Configuring and Installing SSIS on a SQL 2008 R2 failover cluster infrastructure.

SSIS isn’t cluster aware out of the box, so there is some bit of config changes to make in order to make it work on a cluster.

C

Select just the Integration service.

On the Client Access point screen, type in the name and also the IP address as well looking at the requirement spreadsheet.

Click on Next

Since the SSIS service has been installed in its own resource group, we now need to set the dependencies.

Set the disk dependency for the SSIS service

Add the disk resource dependency

Requirement: Change SSIS configuration, custom configuration

SSIS Custom Configuration

Navigate to the path: C:\Program Files (x86)\Microsoft SQL Server\100\DTS\Binn

Create a folder on the root of the SSIS cluster disk called SSIS_Config

Create a folder on the SSIS cluster disk called Packages

Copy the file from the location C:\Program Files\Microsoft SQL Server\100\DTS\Binn\MsDtsSrvr.ini.xml to S:\SSIS_Config\

Right Click the SSIS service and select properties and then go onto Registry replication and add the registry key below.

HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\100\SSIS\ServiceConfigFile\

Click on OK and then Apply.

Take the resource offline and bring it back online for the changes to take effect.

Perform the following steps for SSIS on the second node.

Change the MsDTSSrvr.ini.xml file on the location defined above and change the packages folder to point to S:\Packages

Make the registry entry changes on the second node as well to finish off the configuration.

Test the failover by failing over the SSIS resource group by NodeA to NodeB.

Copy a .dtsx file onto the S:\Packages folder and open up management studio.

In the illustration below, you can see that we have connected to the SSIS instance and we can see the packages.

I was working with SQL 2008 replication and on checking replication monitor, I noticed that on the distribution agent for one of the databases, it had a retrying command status. I further looked into it to see what the problem was, here I can see that the distribution agent had become stucked whilst trying to send commands to the subscriber. The first thing that came to my mind was, “what is happening with the subscriber ? “

I ran the following on the subscriber

use master
GO
sp_who2 active

I was able to see that there is a blocking spid at the moment, which is coming from my distributor. So what must have happened here is that, as a snapshot was being delivered to the subscriber, objects such as stored procedures and functions were also being delivered as well, thereby causing a deadlock to occur.

To confirm this, I can the following

from distributor

Drop SQL server replication after a server name change

After spending some time brushing up my photography skills, its back to the real world of work, coming in to work I had a request from a developer to setup SQL server replication on a publisher with one subscriber, I have a routine of powershell scripts that I normally run to do this. After completing the steps, I noticed some errors, upon investigating it became apparent that the identity of the server isn’t what it is expected to be. The servername is different for some reason.

After running the select command to view the server name,  I find that its different from the server in which I have logged in from.

I get the error

SELECT @@SERVERNAMEsp_dropserver WRONGSERVER

Msg 20582, Level 16, State 1, Procedure sp_MSrepl_check_server, Line 31
Cannot drop server WRONGSERVER because it is used as a Publisher in replication.

–Trying to drop SQL replication I get the following errors

 

exec master..sp_dropdistributor @no_checks = 1

Msg 208, Level 16, State 1, Procedure sp_MSrepl_dropsubscription, Line 103
Invalid object name 'dbo.syssubscriptions'.
Msg 266, Level 16, State 2, Procedure sp_MSrepl_dropsubscription, Line 103
Transaction count after EXECUTE indicates a mismatching number of BEGIN and COMMIT statements. Previous count = 0, current count = 1.
Msg 266, Level 16, State 2, Procedure sp_dropsubscription, Line 0
Transaction count after EXECUTE indicates a mismatching number of BEGIN and COMMIT statements. Previous count = 0, current count = 1.
Msg 266, Level 16, State 2, Procedure sp_MSpublishdb, Line 0
Transaction count after EXECUTE indicates a mismatching number of BEGIN and COMMIT statements. Previous count = 0, current count = 1.
Msg 266, Level 16, State 2, Procedure sp_replicationdboption, Line 0
Transaction count after EXECUTE indicates a mismatching number of BEGIN and COMMIT statements. Previous count = 0, current count = 1.
Msg 266, Level 16, State 2, Procedure sp_dropdistributor, Line 0
Transaction count after EXECUTE indicates a mismatching number of BEGIN and COMMIT statements. Previous count = 0, current count = 1.

In this case, I can tell that there has been an exception within the execution of the stored procedures. What needs doing is for the replication markers on the databases to be removed so that replication can be dropped cleanly on the database.

Step 1

 

-- Remove the replication marker on the databases

EXEC master..sp_MSforeachdb '
USE [?]
IF ''?'' <> ''master'' AND ''?'' <> ''model'' AND ''?'' <> ''msdb'' AND ''?'' <> ''tempdb''
BEGIN
    SELECT ''?''

 IF OBJECT_ID(''tempdb..#dbproperty'') is not null
 drop table #dbproperty

 create table #dbproperty (OptionName   varchar(20), CurrentSetting varchar(10))

 insert into #dbproperty
 exec sp_dboption ''?'',''published''

 IF EXISTS (Select * from #dbproperty where CurrentSetting = ''ON''  )
 exec sp_removedbreplication @dbname = ''?'', @type = ''tran''

 IF OBJECT_ID(''tempdb..#dbproperty'') is not null
 drop table #dbproperty

END
'

This will now drop SQL replication.

exec master..sp_dropdistributor @no_checks = 1

It should now work.

Change the servername now.

sp_dropserver WRONGSERVER

– Create new server name

sp_addserver ‘RIGHTSERVER’,local

Restart the SQL service on the publisher.

The correct new servername should now be reflected on the SQL instance.

Now proceed to setting up SQL replication.

 

SQL Agent alerts not being sent ! Its weird ! Here is the fix for [264] An attempt was made to send an email when no email session has been established

I recently setup some SQL agent alerts but I decided to test them out and make sure they actually work, guess what it wasnt actually working and I wasn’t getting any alerts.

In view of this, I then recreated the datbase mail settings on the database server, I have come across this problem before and the fix then was to restart the SQL agent service and the problem was fixed, but this one was slighly different, I have restarted a few times to avail.

One thing that I did see in the SQL agent error log is as follows.

[264] An attempt was made to send an email when no email session has been established

After checking online and not finding a solution that does fix my problem, I decided to do the following.

Open up management studio >> SQL server agent >> Right click properties >> Alert System

Then untick the Enable mail profile.

Restart SQL agent and click OK.

 

Repeat the steps

Open up management studio >> SQL server agent >> Right click properties >> Alert System

Then tick the Enable mail profile.

Restart SQL agent and click OK.

Setting up WMI query and getting error SQLServerAgent Error: WMI error: 0×80041003.

I was trying to setup database mirroring alerts on one of my production database servers and I looked into the article on the MS site, see http://technet.microsoft.com/en-us/library/cc966392.aspx on mirroring alerting best practices. I then proceeded in trying to setup same, I ran the script below.

USE [msdb]
GO
EXEC msdb.dbo.sp_add_alert @name=N'Mirroring_Alert',
  @enabled=1,
  @delay_between_responses=1800,
  @include_event_description_in=0,
  @wmi_namespace=N'\\.\root\Microsoft\SqlServer\ServerEvents\MSSQLSERVER',
  @wmi_query=N'SELECT * FROM DATABASE_MIRRORING_STATE_CHANGE WHERE State = 5 ',
  @job_id=N'00000000-0000-0000-0000-000000000000'
GO
EXEC msdb.dbo.sp_add_notification @alert_name=N'Mirroring_Alert', @operator_name=N'DBA_GROUP', @notification_method = 1
GO

 

I then got the error message below.

Msg 22022, Level 16, State 1, Line 0
SQLServerAgent Error: WMI error: 0x80041003.
Msg 14511, Level 16, State 1, Procedure sp_verify_alert, Line 300
The @wmi_query could not be executed in the @wmi_namespace provided. Verify that an event class selected in the query exists in the namespace and that the query has the correct syntax.

 

I ran thesame code on my desktop and it was fine, but I was thinking what could be the problem here.

I checked everything to make sure that the syntax was right and there wasnt anything wrong, bumped across a few KB’s about thesame problem, but it was primarily being caused by server names being longer than 15 characters and this applied to SQL 2005, my SQL instance was SQL 2008 and I then thought what could be going wrong. After spending the hour battling this, and I have a rule that any problem taking longer than an hour to fix, I need to get my head cooled out, pop out for a 10 minute break and fresh air and then give it another go. It seem to work for me time and time again, I came back, thought !! Hang on a minute, lets check the SQL error log to see if there are any pointers to the root cause, on there I saw errors as follows.

Date  6/13/2011 11:45:33 AM
Log  SQL Server (Current - 6/13/2011 10:01:00 AM)

Source  Logon

Message
Login failed for user 'NT AUTHORITY\SYSTEM'. Reason: The account is disabled. [CLIENT: <local machine>]

Ahh, this could be it. When I setup SQL servers I often disable the builtin/Administrators login as this is a security risk. It looks like the WMI service is using the local system account and this is obviously disabled and thus causing the problem highlighted.

I changed the WMI service account such that its a domain account and permissioned it on SQL server and pesto !! The problem is solved.

Be Careful ! SSAS Analysis services cannot be uninstalled from a SQL cluster once installed

SQL 2008 Management Datawarehouse Overview

 

With the release of SQL 2008, Microsoft has developed a warehouse framework for database administrators to have an idea of what is actually going in within SQL server. A lot of the times during my career, I have heard the phrase “The database was slow between 3AM and 4AM) This is especially true if you work within a global framework whereby the database server is serving requests 24/7. I would have users in the Asia pacific region using the database from around midnight GMT (UK time), the as they finish the UK users start around 8AM, and as they wrap up the US users kick in around 1PM BST. So it’s a never ending circle.

To buttress the point, management data warehouse gives me the high level and detailed view of what is happening on the database server, I like to use the waits and queues methodology to see what SQL server is waiting for, and to also keep tabs on my queues as well. If a process is queing up waiting for resources, then there must be a reason for it and as a database administrator, I want to check and understand why we have the waits and queues on the database server.

This post assumes that management data warehouse has already been installed on the database server, I am still running SQL 2008 RTM SP1 cumulative update 5 on my systems and till date, there is no way to remove management data warehouse once it has been installed. I installed management data warehouse on the dedicated DBA database on the production system, the database schema gets installed on the database as well.

To launch management data warehouse, follow the steps below.

Management data warehouse works by collecting information from a number of DMV’s and using connection points to upload the data onto the database and then store it within  data warehouse style tables.

 

Memory section

 

If you click on the memory graph, it drills down to show you the key memory counters. Here you can see things like page life expectancy, SQL server internal memory consumption by type

Reviewing waits, within management data warehouse you can check your wait statistics and further drill into what percentage the composition is.

Memory counters based on page life expentancy.

 

 

 

 

 

Disk IO statistics information can also be found below.

On the IO specific information, you can drill down on top queries based on duration, total IO, physical disk reads and also logical writes.

Move System model database in SQL 2008 R2 Failover Cluster

Move System database on a SQL 2008 R2 Failover Cluster

In order to do this, run the following steps

• Modify the location of the system databases
• Open up the failover cluster manager.
• Run the code below with the SQL resource online

Alter database msdb modify file (name=msdbdata,filename='F:\Database\msdbdata.mdf')

Alter database model modify file (name=modellog,filename='L:\Logs\modellog.ldf')

• Bring the SQL resource offline

• Confirmation from the failover cluser manager

• The resource has now been taken offline

• Now move the files onto the new destination, i.e the location of where the system data and log file should be.
• Bring the SQL service and sQL agent resource online.

SQL Failover Clustering Requirements

When installing a SQL failover cluster, its a complicated process and we DBA’s are always looking for ways to make our lives easier. Also as a SQL failover installation would require cooperation between the following groups of people.

1. Database Administrators
2. Windows Administrators
3. Network Administrators
4. Storage Administrators.

I find that creating a small little spreadsheet and sending it around for everyone to fill in their details or perhaps state your requirements often helps and avoids delays and ambiguity.

I have put together a spreadsheet with same information as it might be of use to others.

SQL Failover Cluster_Requirements_v1.0

SQL 2008 Failover Cluster Installation Pre-Staging

As a DBA, I can find myself cross between being able to do my job and also some kind of wrangling with system administrators, they seem to believe that anything to do with installations ought to be the work of a system administrator. I do not believe that life has to operate in that fashion. I think DBA’s should be allowed to do what they are very good at and what they were hired for and system administrators should focus on core operating system tasks. I have encountered crossroads which has led to disagreements etc, once I had a system administrator saying that a project would be stalled unless he installed SQL 2008.

Anyway, away from the ranting. I had to install and configure a SQL 2008 R2 failover cluster the other day, and ran into some difficulties when it came to registering the virtual SQL network name in Active Directory, of course I am not a domain administrator. I do not need to be a domain administrator to do my job. On speaking to one of the system administrators, he seemed to think that it was the job of a system administrator to install SQL on a failover cluster and a DBA can sit back and watch. I disagree, I think DBA’s should work with system administrators and work together but yet focus on areas in which they specialise.

I managed to speak to the head of the windows administrators, who agreed to pre stage the windows cluster name for me in Active directory, so that the installation can go ahead. This was a sensible approach to me, after all system administrators have already clustered the underlying windows operating system and DBA’s can take a back seat when it comes to that, SA’s on the other hand should allow DBA’s focus on SQL server which is their area of specialism. The windows clustername and the SQL network name were pre-staged in active directory following the steps below.

The specific error message appearing is as follows

Cluster network name resource ‘SQL Network Name (SQLCLUSTER1)’ failed to create its associated computer object in domain ‘domain.local’ for the following reason: Unable to create computer account.
The text for the associated error code is: Access is denied.

Please work with your domain administrator to ensure that:
- The cluster identity ‘WIN-CLUSTER-1′ can create computer objects. By default all computer objects are created in the ‘Computers’ container; consult the domain administrator if this location has been changed.
- The quota for computer objects has not been reached.
- If there is an existing computer object, verify the Cluster Identity ‘WIN-CLUSTER-1$’ has ‘Full Control’ permission to that computer object
–2
Cluster resource ‘SQL Network Name (SQLCLUSTER1)’ in clustered service or application ‘SQLCLUSTER1-SQL’ failed.

–3
The Cluster service failed to bring clustered service or application ‘SQLCLUSTER1-SQL’ completely online or offline. One or more resources may be in a failed state. This may impact the availability of the clustered service or application.

To prestage a cluster name account

1. Make sure that you know the name that the cluster will have, and the name of the user account that will be used by the person who creates the cluster. (Note that you can use that account to perform this procedure.)
2. On a domain controller, click Start, click Administrative Tools, and then click Active Directory Users and Computers. If the User Account Control dialog box appears, confirm that the action it displays is what you want, and then click Continue.
3. In the console tree, right-click Computers or the default container in which computer accounts are created in your domain. Computers is located in Active Directory Users and Computers/domain node/Computers.
4. Click New and then click Computer.
5. Type the name that will be used for the failover cluster, in other words, the cluster name that will be specified in the Create Cluster wizard, and then click OK.
6. Right-click the account that you just created, and then click Disable Account. If prompted to confirm your choice, click Yes.

The account must be disabled so that when the Create Cluster wizard is run, it can confirm that the account it will use for the cluster is not currently in use by an existing computer or cluster in the domain.

1. On the View menu, make sure that Advanced Features is selected.

When Advanced Features is selected, you can see the Security tab in the properties of accounts (objects) in Active Directory Users and Computers.

1. Right-click the folder that you right-clicked in step 3 , and then click Properties.
2. On the Security tab, click Advanced.

10.  Click Add, click Object Types and make sure that Computers is selected, and then click OK. Then, under Enter the object name to select, type the name of the computer account you just created, and then click OK. If a message appears, saying that you are about to add a disabled object, click OK.

11.  In the Permission Entry dialog box, locate the Create Computer objects and Read All Properties permissions, and make sure that the Allow check box is selected for each one.

12.  Click OK until you have returned to the Active Directory Users and Computers snap-in.

13.  If you are using the same account to perform this procedure as will be used to create the cluster, skip the remaining steps. Otherwise, you must configure permissions so that the user account that will be used to create the cluster has full control of the computer account you just created:

1. On the View menu, make sure that Advanced Features is selected.
2. Right-click the computer account you just created, and then click Properties.
3. On the Security tab, click Add. If the User Account Control dialog box appears, confirm that the action it displays is what you want, and then click Continue.
4. Use the Select Users, Computers, or Groups dialog box to specify the user account that will be used when creating the cluster. Then click OK.
5. Make sure that the user account that you just added is selected, and then, next to Full Control, select the Allow check box.

In our case, we specifically followed the step below.

Steps for prestaging an account for a clustered service or application

It is usually simpler if you do not prestage the computer account for a clustered service or application, but instead allow the account to be created and configured automatically when you run the High Availability wizard. However, if it is necessary to prestage accounts because of requirements in your organization, use the following procedure.

Membership in the Account Operators group, or equivalent, is the minimum required to complete this procedure. Review details about using the appropriate accounts and group memberships at http://go.microsoft.com/fwlink/?LinkId=83477.

To prestage an account for a clustered service or application

1. Make sure that you know the name of the cluster and the name that the clustered service or application will have.
2. On a domain controller, click Start, click Administrative Tools, and then click Active Directory Users and Computers. If the User Account Control dialog box appears, confirm that the action it displays is what you want, and then click Continue.
3. In the console tree, right-click Computers or the default container in which computer accounts are created in your domain. Computers is located in Active Directory Users and Computers/domain node/Computers.
4. Click New and then click Computer.
5. Type the name that you will use for the clustered service or application, and then click OK.
6. On the View menu, make sure that Advanced Features is selected.

When Advanced Features is selected, you can see the Security tab in the properties of accounts (objects) in Active Directory Users and Computers.

1. Right-click the computer account you just created, and then click Properties.
2. On the Security tab, click Add.
3. Click Object Types and make sure that Computers is selected, and then click OK. Then, under Enter the object name to select, type the cluster name account, and then click OK. If a message appears, saying that you are about to add a disabled object, click OK.

10.  Make sure that the cluster name account is selected, and then, next to Full Control, select the Allow check box.

There is an excellent kb article which can be found on the link below.

http://technet.microsoft.com/en-us/library/cc731002(WS.10).aspx

 References: http://technet.microsoft.com/en-us/library/cc731002(WS.10).aspx