Today, one of my SQL Server Magazine articles went live and its topic was an interesting one…‘Is SQL Server 2008 BI worth an upgrade?’ I have had the great pleasure of working in a large variety of IT environments over the years and one trend I’ve noticed becoming more prevalent is organizations implementing Microsoft SQL Server just for Business Intelligence. By no means am I implying that SQL Server is not a great OLTP (relational database engine) as well. The facts though are that Microsoft has invested (and continues to invest) a great amount of resources into its overall platform’s BI capabilities. Some of you may have heard the informal nickname for SQL Server 2005, ‘the BI release’. These investments reflect the current status of corporate IT environments which are increasingly adopting|implementing more Business Intelligence based systems.

Microsoft SQL Server 2008 continues the trend of improving its BI capabilities. Every major BI component (including the relational engine) has been enhanced in terms of both performance (scalability) and capabilities (features). Two other items I want to mention quickly (neither of which are in the corresponding article) is that of Change Data Capture (CDC) and Resource Governor (RG). CDC is a great feature for enabling your SQL Server 2008 based OLTP systems (yes, you would have to upgrade your OLTP servers) to expose their changed data asynchronously for ETL consuming purposes! In addition, RG can be used to prioritize OTLP workloads including that of ETL processes, this makes it easier to ensure that BI requirements do not adversely affect the more important OLTP-based applications (business transactional systems). So the big question still remains…is SQL Server 2008 BI worth the cost (and effort) of an upgrade? Unfortunately, like most difficult questions in life the answer is that it depends upon your unique business intelligence environment and requirements!

Cheers

Derek Comingore

Like Joe, things have been quite busy on my end as well (which is good). I have been doing alot of research as of late in the Microsoft SQL Server Reporting Services (SSRS) 2008 visualizations. With the arrival of SSRS 2008 users will now gain out-of-the-box access to the Dundas visualizations platform. If your curious, the initial Dundas press release can be found here. SSRS 2008 (as of the February 2008 CTP build) contains both Dundas Gauge and Dundas Chart products. In addition, according to the prior mentioned Dundas press release it states that Dundas Calendar will also be included into SSRS 2008.

This fairly recent Microsoft purchase of the Dundas source code for integration into SSRS 2008 is a great move as the Dundas suite of SSRS add-ons have become the premier choice for such advanced visualization needs. By including the Dundas technologies into SSRS 2008, Reporting Services customers will not only gain access to a much improved Report Server Architecture (without requiring IIS) but also an enhanced Visualization platform. There are literally about three times as many chart types in SSRS 2008 as compared to SSRS 2005. Some of the brand new chart types include the Funnel, Range, Pyramid, and Polar. In addition to the added chart types customers will also gain access to the Dundas Gauge capabilities via a new Gauge Data Region. As of the latest SQL Server 2008 CTP there are two main gauge types: Radial and Linear. Finally, we get a few other ‘goodies’ with the inclusion of the Dundas suite including:

• Secondary Axes
• Runtime Calculated Series
• WYSIWYG Chart Editor (design-time)

Below are some sample screenshots of the new SSRS 2008 visualization capabilities…

Cheers

Derek Comingore

Been away for some time finishing up on the Upgrade and Application Compatibility testing content and tools. We’re about 98% done with just a few more tweaks before final RTM. If you’re interested in the release candidate versions, check out SQL Server 2008 Upgrade Assistant.

Managed to find some time for an extended lunch so I figured I’d come back and share some other learnings - an easy one this time: backup compression. This is not the same as data compression so don’t confuse the two. I’ll blog how to on the latter later when I find some time. That’s pretty cool too.

I’ve worked with a lot of customers who use 3rd party backup software primarily for the compression capabilities followed by improved speed, especially during restore. With one of the more popular 3rd party backup solutions, I’ve restored a 1.2 TB database in under 30 minutes on a beefy HP Integrity rx8640 server with very a large EMC Dx3 SAN so there was no IO or CPU bottlenecks at any point. No official hard data on Katmai’s backup compression or performance right now but I know the folks in building 35 will be working hard to at least match the popular 3rd party solutions.

From some of the tests we’ve done both internally and with a couple of customers with tens and hundreds of GB databases, both compression ratio and performance seem to map closely to the popular 3rd party software we used. At least for default compressions on the 3rd party software. You can change the level of compression with some of these 3rd parties but SQL Server 2008 only has 1 level so I didn’t bother testing against different settings. I’d imagine the delta would not be significant percentage wise but if you have tens of Terabytes to deal with, even 5% can be a noteworthy difference.

How it’s done

Taking advantage of backup compression once you’ve upgraded to Katmai is just a matter of adding an extra option (in bold below) in your backup scripts 

– My Backup Job with compression 

BACKUP DATABASE MyBigDatabase
TO  DISK = ‘N:\Tier1Backups\MyBigDatabase.bak’
WITH
NOFORMAT,
NOINIT,
NAME = ‘Full DB Backup - MyBigDatabase’,
SKIP,
NOREWIND,
NOUNLOAD,
COMPRESSION;
GO

or selecting the compression option if you are using Management Studio GUI.  Note that you can change the global setting at the instance level with

SP_CONFIGURE ‘backup compression default’, 1;

so that all backup commands will always compress by default. This can be changed at the command level also when you execute your backup so you can backup with compression for specific databases only and not for everything in the instance. This is too easy.

Note that the default setting is ‘no compression’ so your DBA team does not get alarmed by the sudden reduction in the backup file sizes; just in case not everyone is keeping close tabs Katmai enhancements. Last thing you want is the graveyard shift operator re-running backup all night and finally paging you at 3am because “the backup file looks too small so it can’t be good…”. Don’t laugh (unless it’s someone else’s turn to be on-call).

Sample results

Simple tests with the AdventureWorks OLTP database yielded the following:

Path	Backup time	Restore time	File size
Original database	n/a	n/a	174,080 MB
NO COMPRESSION backup	23s	23s	171,093 MB
WITH COMPRESSION backup	11s	14s	40,097 MB

This was done on a VPC with 1.5GB of dedicated RAM and its own dedicated 7200rpm EIDE disk (8MB cache) on a USB 2 interface. The host machine is a notebook with an Intel Core 2 Duo 2.2GHz CPU (T7500), 4GB RAM and a 7200rpm SATA disk. The times are based on averages from 3 runs - all were with a couple of seconds. Other tests with real customer databases ranging from 5GB to 800GB in size yielded similar levels of compression and percentage elapsed time reduction. Sorry, can’t publish any of the customer data but those tests were run on x64 workstations with local SATA drives and IA64 servers with enterprise class SANs.

Not bad at all. Note the benefits from a performance perspective since you have much less to send to disk or retrieve when restoring. There was some impact on CPU utilization but it’s hard to say what the real impact would be when doing hot backups in a production environment since I don’t have other workloads running. You can expect some CPU load but the level will really depend on your database. Looks like it’ll be low to moderate. So long as you’re not already facing CPU pressure in your current system, I expect this to work fine but don’t take my word for it; test (no really, test). This is important because while few environments do full DB backups during peak hours, many companies have other system jobs or reports run during maintenance windows so make sure you don’t saturate the CPU. All that aside, the numbers are very decent especially since it’s still in beta.

Of course, with any compression technology, database or file system, your mileage can vary significantly. Character data types typically compress very well while binary types not as well (or very poorly in some cases). However, most traditional OLTP systems (e.g. accounting, sales, HR, etc…) will probably benefit from this since the bulk of the data is of character types.I expect this will be a widely used feature both to improve backup/restore performance and to save disk space (enterprise storage is NOT cheap).

One thing to remember is that smaller backup files also benefit some disaster recovery scenarios (basically those that don’t have good DR plans). Companies that do not have a SAN replication solution or some kind of Log Shipping/Database Mirroring solution in place, often transfer backup files to their warm DR site. Smaller files means faster transfer time which means better chance of recovery (you have a smaller window of exposure).

Password lock is available as with regular backups but there is no encryption capability with this release as far as I know. Now if you’re thinking, “hey, I’ve got Transparent Data Encryption, I can mix and match!!“. Well, don’t. Backup compression and TDE, at least based on the current release, do not compliment each other. You get next to nothing in space savings with backup compression on a TDE enabled database but you will burn the extra CPU cycles trying. As far as I can tell, this is true regardless of whether you use SQL Server’s backup compression or some 3rd party backup software with compression. However, some 3rd party solutions do offer backup compression with encryption but that’s a whole different deal. It’s a compressed backup that’s stored encrypted on disk with a non-TDE database.

Chalk this up as one of the “little to no effort” benefits when you move to SQL Server 2008. Just be aware that while it does great at the intended task, it doesn’t have all the bells & whistles that popular common 3rd party solutions do. For example, having different levels of compression and encrypting backups of non-TDE database are the most common I hear from our customers. Got something you want the SQL Server product group to add something for the next release? Go to SQL Server Connect

joe.

Recently I gave a presentation on Reporting Services 2008 for the information worker. This webcast focuses mainly on the new Report Designer tool utility that ships with SQL Server 2008 that is designed for business users. The presentation can be found HERE.

Cheers

Derek Comingore

Analysis Services 2008, the Next-Generation Analytics Platform

Since Microsoft’s 1996 acquisition of the Israeli-based Panorama Software, Microsoft’s Business Intelligence platform has been continually refined and is spearheaded by the later renamed Analysis Services analytical platform! Analysis Services continues to rank as the number one OLAP product according to such institutes as Gartner, IDC, and the OLAP Report. With each iteration that passes Analysis Services continues to ‘up the ante’ in the OLAP market.

Analytical platforms in general are growing and are realizing more mainstream use. Every potential business and industry can benefit from analytics and thus the larger field of Business Intelligence! Competition is fiercer today in business than ever before and the ability to reap more value from your existing corporate data then your competitors is one of the many ‘winning factors’ in business today!

Every major area of the SQL Server product has been enhanced in the 2008 release including Analysis Services. The new enhancements as described later touch both multidimensional and data mining objects. Analysis Services 2008 improvements are substantial and thus warrant an evaluation of Analysis Services 2008 and the potential benefits it may bring to your organization!

Why Upgrade from Analysis Services 2000?

The same reasons for upgrading to Analysis Services 2005 from 2000 apply to why you should consider upgrade to Analysis Services 2008. In addition to those original improvements found in the 2005 release, 2008 contains additional improvements which are listed below in the topic of Why Upgrade from 2005. Beyond the analytical platform improvements there is the end of mainstream support for SQL Server 2000 matter. Microsoft mainstream support for SQL Server 2000 ends on 4/8/2008, more information can be found on this at http://support.microsoft.com/lifecycle/?LN=en-us&p1=2852&x=17&y=10 .

Why Upgrade from Analysis Services 2005?

Like the DTS to SSIS 2008 migration, if you are already running on the 2005 platform you will realize less benefits when compared to those migrating from Analysis Services 2000. That being said, there are still a good amount of new features and improvements found in Analysis Services 2008 including but not limited to:

• MultiDimensional Data
  • Enhanced Cube Design
    • Simplified & Enhanced Cube Wizard ‘IntelliCube’
    • AMO & UI Best Practice Alerts
    • Aggregations Designer
    • Simplified & Enhanced Aggregations Designer & Usage-Based Optimizations Wizards
  • Enhanced Scalability
    • MOLAP-enabled write back
    • Scale-out databases
    • Backup Compression
  • Enhanced Dimension Design
    • Attribute Relationships Designer
    • Enhanced Dimension Wizard
    • Key Columns Dialog
  • Enhanced MDX
    • Dynamic Named Sets
    • Subspace Computations
• Data Mining
  • ARIMA Time Series Algorithm
  • Creation of Holdout Test Sets
  • Filtering on Model Cases
  • Cross-validation of multiple Mining Models
  • Drill though to Structure Cases & Columns
  • Aliasing Mining Model Columns

As you can see there are still several added benefits by migrating to Analysis Services 2008! Microsoft continues to devote a large amount of its resources to continual refining the coveted MSFT BI platform. So should you upgrade to Analysis Services 2008? Ultimately, this decision should be based on your organization’s analytical applications and use of Analysis Services and the degree you can benefit from the new capabilities of the 2008 platform.   

We are still running on OLAP Services (SQL 7.0), what do we do?

Things get a bit more difficult if your organization is still running on the original SQL 7.0 OLAP Services release! First, you will need to upgrade to one of the two supported SQL Server 2008 upgrade paths:

1.       SQL Server 2000 w/sp4

2.       SQL Server 2005 w/sp2

Once you have performed this preliminary upgrade of your SQL Server (and thus OLAP Services) server you can start planning your SQL Server 2008 upgrade if you desire to get the latest advances in the analytics platform! Note that mainstream support for SQL Server 7.0 concluded on 12/31/2005! For more information on Microsoft’s Support Lifecycle for SQL Server 7.0 please see http://support.microsoft.com/lifecycle/?p1=2862 .

Effort Estimation Phase

Yet another parallel with the DTS to SSIS 2008 migration, you should consider performing a migration effort inquiry phase. There are fairly large differences between the 2000 version of Analysis Services and those versions that followed thereafter. Some of the items you may want to consider when performing your Analysis Services migration effort estimation include but not limited to:

·         Cube & Mining Structure/Models Complexities

·         SQL Server Upgrade Advisor (SSUA) 2008’s alerts on your existing Analysis Services Installations

·         64-bit Environments

Summary & Part2

In this post we have reviewed why an organization should consider upgrading their existing Analysis Services (and OLAP Services) installations to Analysis Services 2008. Much like the DTS to SSIS 2008 migration, Analysis Services migrations (particularly from SQL Server 2000) are a somewhat difficult yet high value proposition. A second parallel is having a solid, well thought out migration plan and effort estimation phase completed. Part 2 of our Analysis Services 2008 migration topic will go into the actual migration itself, rather how we perform the migration once we have decided to go forward!

Cheers

Derek Comingore

Time to Upgrade…

Well, we’re back and it’s time to perform the actual DTS to SSIS migration. Before we get started let’s review a few key points of interest. First, the migration process from DTS to SSIS is the same regardless of which version of SSIS (2005 or 2008) you are migrating too. In the context of these blog posts we are discussing SSIS 2008, just keep in mind that these are the same processes to migrate to SSIS 2005 as well. Second, we need to review the available DTS to SSIS migration strategies as mentioned in my first post, The Key SQL Server 2008 BI Migration: DTS to SSIS 2008, Part 1 of 2:

·         Proactive (resolving DTS to SSIS issues prior to upgrade)

·         Reactive w/Package Migration Wizard

·         Reactive w/o Package Migration Wizard

There are various business and technical requirements that will assist you in this decision such as:

·         Can you still work in/edit the DTS Packages?

·         Can the majority of the DTS to SSIS issues be resolved in DTS or must you wait until the package is in SSIS format? (some DTS items do not have direct SSIS equivalents)

·         What is the average effectiveness of the Package Migration Wizard for your packages?

The DTS to SSIS Migration Process

At this point you have been given the ‘green light’ to perform the actual migration and you have selected a migration strategy. So now what? Well, first off don’t do anything on production! If you have not yet performed a DTS Package scan with SQL Server Upgrade Advisor (SSUA) for the effort estimation, that will be the first task to perform. Again, we want to minimize the load on the production server(s) as they still have to do their daily ‘jobs’. To perform a SSUA scan of your DTS Packages perform the following steps:

1.       Export the DTS Packages to the Structured Storage File Format.

a.       In SQL Server 2000 Enterprise Manager, open each package and do a Save As command to the Structured Storage File Format.

*If the production server has already been upgraded to SQL Server 2005 and is still running the DTS Packages you will need the SQL Server 2000 DTS Designer Components to save the package to a storage file.

*You can save each package to a new file or consolidate all of them into a single file. The SSUA facilitates scanning all files in a supplied directory so either method will work with the SSUA.

2.        Copy or Move the files to a SQL Server 2008 development server.

3.        Scan the DTS Packages w/SSUA 2008

*The SSUA requires the detection of the DTS runtime which gets installed as part of the SQL Server 2005|2008 backwards compatibility files prior to allowing you to scan individual .dts (structured storage) files.

4.        Note the SSUA scan results

With your SSUA scan results ‘in hand’ let’s move on. You should already have the DTS packages on a development server. At this point you should either use your current development server or another server to create a true test environment. By test environment I mean a SQL Server 2008 environment that mimics production. So items like files, directories, databases (with a sampling of data) etc are on the development server, this will facilitate the actual testing of your SSIS 2008 packages after being converted!

At this point our development environment has been built using SQL Server 2008. It is now time to migrate the packages in the development environment by leveraging your chosen migration strategy detailed above. Once you have your SSIS (.dtsx) equivalents of the DTS packages you need to perform testing of each individual SSIS package. Make sure they execute as expected producing the correct results. Once you are confident that the new SSIS packages are working as expected you should:

·         Enable SSIS Package Configurations for all external resources

·         Create a Package Deployment Utility

You are now ready to deploy, test, and ensure that the new SSIS packages run as expected in your production environment! Definitely coordinate the following steps with the server’s DBA. Depending upon the overall SQL Server 2008 upgrade strategy the DBA choose (in-place or side-by-side) you may or may not have both instances on the current production machine. If the production servers have not yet been upgraded then you must wait until they are upgraded to SQL Server 2008. Once the production server(s) are upgraded, deploy the packages with the previously created Package Deployment utility to the intended servers. Update any package configuration settings needed and test each individual package during periods of low activity on the production server (if possible). Once you have confirmed the new SSIS Packages work as expected your DTS to SSIS 2008 migration is complete!

Migrating from SSIS 2005 to 2008

If you are already running on SQL Server 2005’s Integration Services platform and you wish to get the new benefits of Integration Services 2008 you will still need to perform a migration (or upgrade), however it’s a relatively simple one. As of the time of this writing (SQL Server 2008 CTP5), SSIS 2008 does support being installed side-by-side with SSIS 2005. SSIS 2008 is for the most part a pure enhancement of the platform. I say for the most part because there are a few ‘gotchas’ (breaking changes) between the two builds:

·         SSIS 2008 uses Visual Studio 2005 Tools for Applications (VSTA) for its scripting environment

·         SSIS 2008 uses a newer build of the SQL Server Native Client

·         SSIS 2008 contains an updated OLEDB Provider for Analysis Services (10.0)

·         SSIS 2005 custom developed components will need to be altered to work in SSIS 2008

The script environment differences are the biggest change between the two builds of SSIS. When you open/add a SSIS 2005 package which contains a Script Task in SQL Server 2008 BIDS it will automatically attempt to convert the script for you. You can also attempt to execute the older SSIS 2005 package with the 2008 dtexec utility, this operation will also convert the script; however the script conversion is only temporary for the currently running instance of the package.

Removing the old DTS Packages & SQL Server Agent Jobs from Production

This is the step that is the easiest to forget about! You have now successfully migrated your older DTS packages to the SSIS 2008 format, we are all done right? No, that’s not right, we still have the original DTS packages running on our production server. To remove the older packages from a production environment perform the following steps (again make sure the DBA is aware of your actions):

·         Stop all corresponding SQL Agent jobs

·         Open each package and do a Save As command to export the production version to the structured storage file format (yes, I know we already did this, but you never know if any of them changed or not, good practice)

·         Backup, check in, etc the storage files just in case we ever need to refer to them again!

*Assuming the same schedules apply to the new packages you may want to just update each SQL Server Agent Job with the corresponding SSIS Package step replacing the older DTSRun steps. If you need to recreate the jobs too on a different server consider scripting out all of the jobs and then editing/recreating them in bulk with the updated job steps in the scripts.

Summary

In this post we have explored the various DTS to SSIS migration solutions and best practices processes. Once the migration is approved based upon the business and technical factors (see my prior post The Key SQL Server 2008 BI Migration: DTS to SSIS 2008, Part 1 of 2) you need to perform the actual migration. The DTS to SSIS migration summarized consists of:

1.       Choosing a DTS to SSIS Migration Strategy (Reactive/Proactive)

2.       Capturing SSUA DTS Package Alerts (all categories of notifications)

3.       Building a dev/test environment

4.       Migrating the packages using the selected DTS to SSIS Migration Strategy

5.       Testing/Correcting the resulting SSIS 2008 Packages in the dev/test environment

6.       Deploying and reconfirming the resulting SSIS 2008 Packages work in production as expected

7.       Removing the old DTS Packages from production w/optional SQL Server Agent Jobs

By using the prescriptive guidance found in both of my posts you will have a great head start on the DTS to SSIS migration process we all face as SQL Server customers!

Cheers

Derek Comingore

We have had a lot of great posts from contributing author Derek Comingore regarding application compatibility and upgrades of the key SQL Server 2008 components that sit outside the relational engine. In this entry, I’m posting a podcast interview we created with Derek that covers many of these points so our podcast subscribers can get the content as well.

Fire up those Zune’s and enjoy!

 

 SQL Server 2008: BI Stack Key App Compat Issues [16:58m]: Play Now | Play in Popup | Download

-Derick

The DTS to SSIS Dilemma…

SQL Server Integration Services or SSIS, saying those words around IT personnel who have a plethora of DTS packages will shiver! Naturally, they tremble because of two key reasons. First, SSIS is a completely different and more powerful tool, with power does come a certain degree of complexity. DTS has been used for a range of tasks beyond simple Extract, Transform, Load (ETL) purposes. Recently I was at a client’s site that entailed over 400 DTS packages, few of which were used for actual decision support purposes. Analyst, Information Workers, and others that are common non-IT personnel users of DTS are going to feel awkward (at least at first) using SSIS!

The second reason that folks get nervous about any potential DTS to SSIS migrations is because they are to some degree aware of the migration ‘cost’ or effort involved. DTS to SSIS is not a trivial migration, especially for those clients who have the 100s or even 1000s of production packages. There is a package migration wizard I will discuss in part 2 of this post that ships with SQL Server 2005 and 2008.  The package migration wizard attempts to migrate your DTS packages, however this wizard roughly migrates only around 40% of the various possible package configurations!

So Why Migrate Given the High Effort Required?

This is a natural question and the answer is also simple: power and speed. First, SSIS is a much more powerful data integration platform. SSIS contains several features that the older DTS platform did not have. Here are some of the features of SSIS that DTS did not have (or at least to the degree that SSIS contains):        

• Separation of control flow and data flow        
• Some connections have been optimized        
• Several additional various transformations        
• Wider range of data sources        
• Security enhancements        
• Toolset unification via BIDS with the other BI projects, source control integration
• Checkpoints & in place restarts        
• Dynamic runtime configurations        
• Superior logging capabilities 

The second reason to migrate is speed or rather performance. SSIS entails two different yet cooperating engines, the Run-Time Engine and the Data Flow Engine. SSIS’s Data Flow Engine leverages a buffer-oriented architecture to load and transform data in memory. As a result, SSIS is fairly fast compared to DTS, especially in terms of transformations. If you are performing simple data loads without many transformations involved then there will be a small delta in performance leaning towards SSIS (SSIS’s SQL Server destination component is faster than the older DTS connections). That said, as you increase package transformation complexity, SSIS performance will continue to increase over DTS. In fact, on average SSIS is believed to be roughly 7 times as fast as DTS. Your organization may experience a lesser increase in performance or even higher but this is the average gain seen.

A third reason though it is not a prevalent one yet (which is why I did not mention it before) is because the DTS product is an official deprecated feature of SQL Server. What this means is that Microsoft is delivering a message to its customer that they need to migrate to the SSIS platform sooner than later. Some would argue with me that this should be the main reason why organizations should migrate but let’s be honest about the matter. SQL Server 2008 is the last version to support DTS (as of now) and thus as a customer you will have a long time until you are absolutely forced into this migration but note that the day will come.

We Are Already Running on SSIS 2005, Do We Need SSIS 2008?

This is where things get a bit more difficult. Do you really need to perform an upgrade to SQL Server 2008 just for the SSIS enhancements? Like most things in life, that will depend on your organization’s usage of SSIS and the larger SQL Server product. Product enhancements beyond SSIS are out of scope for this blog post but suffice it to say there are several BI related enhancements in the 2008 product beyond those found in SSIS. That said, below are the enhancements from SSIS 2005 to 2008:        

• New Data Profiling Task & Viewer        
• Enhanced Performance and Caching for the Lookup Transformation        
• Enhanced Data Flow Engine Tuned for Parallelism       
• Various smaller enhancements for supporting new data types and TSQL capabilities

OK, I Want SSIS…How Do We Migrate?

There are three core approaches to the DTS to SSIS 2005|2008 migration. I will be listing these solutions in order of ease. Also, only the last two are possible solutions if you must migrate immediately for some reason.·        

• Proactive

SolutionBased on the SQL Server Upgrade Advisor’s results, take corrective action on the existing DTS packages then re-scan them w/Upgrade Advisor. Once the Upgrade Advisor reports no outstanding issues leverage the 2005 Package Migration Wizard to automate the package migration process. Any packages that cannot have their upgrade issues resolved will need to be rebuilt from scratch in SSIS. 

• Reactive Approach w/Package Migration Wizard

Disregard the Upgrade Advisor’s alerts and use the Package Migration Wizard to perform the migrations. Review each new resulting SSIS package to ensure it is built correctly. For those packages which did not migrate correctly either modify the resulting SSIS package or create it from scratch in the SSIS designer. 

• Reactive Approach w/o Package Migration Wizard

Disregard the Upgrade Advisor’s alerts and rebuild all of the packages from scratch in the SSIS designer. 

Scalability Experts DTS to SSIS Effort Estimation Framework & Considerations

The first step in performing a DTS to SSIS migration project is the effort estimation itself. This is not a trivial migration process and thus warrants a short yet focused study on the effort required. At Scalability Experts we have developed a DTS to SSIS Effort Estimation Framework for such projects. The framework assists us in both the time and accuracy of our DTS to SSIS effort estimations. When you are performing a DTS to SSIS effort estimation some of the items you want to consider include:

• Package Complexities
• Package Upgrade Advisor Alerts
• Parent/Child Package Environments
• Package External Dependencies
• 64-Bit Environments

Summary & Part 2

In conclusion, DTS to SSIS is a somewhat difficult yet highly valuable migration for those organizations that have come to rely on the Microsoft data integration platform(s). With the arrival of SSIS 2008 there are even more reasons to perform the migration. Having a solid migration planned out and effort estimation phase completed will be key steps to ensuring a smooth migration. In part 2 of this blog post we will discuss the actual migration process itself (How to perform the upgrade). This is the next logical step in the process once management has given the thumbs up for the migration effort to continue. Cheers
Derek Comingore

In this post I want to take a break from analyzing the in-depth SQL Server Business Intelligence (BI) upgrade issues facing SQL Server customers and talk about the upgrade process at a higher level. This post could logically be grouped with Joe’s prior post on the 2000 to 2005 or 2008 question facing SQL Server customers. Thanks Joe for getting my mind ‘running’ on this larger upgrade issue and the commonality that exists for SQL Server BI customers upgrading to the 2005 or 2008 versions.

So what is this ‘heavy lifting’ I am referring to? The ‘heavy lifting’ represents the major infrastructure changes that occurred between SQL Server 2000 and 2005 in the Business Intelligence stack. The APIs that power all of the User Interfaces (UIs) in SQL Server (and facilitate your own custom administration applications) were changed and complimented. SQL Server Distributed Management Objects (DMO) was complimented with a new relational admin object model called Server Management Objects (SMO). SQL Server SSAS’ (SSAS) Decision Support Objects (DSO) object model was complimented with a new analytical admin object model called Analysis Management Objects (AMO). Additionally, for programmatic replication admin Replication Management Objects (RMO) was added to the 2005 product. If you are curious I use the term ‘complimented’ because the older object models still shipped with the 2005 & 2008 products. The newer object models are managed (CLR/.Net based) and expose the new features, while the older object models continue to support the older feature sets and are COM-based.

*In SQL Server 2008 Books Online CTP5 there are several instances mentioning SQL Server 2008 will be the last version of the product to entail the older COM-based object models. Plan to port your scripts and admin applications that leverage these older object models to their newer managed equivalents!

We saw the release of SQL Server Integration Services (SSIS) with the 2005 release which required either a manual migration or a partial-manual migration using the Package Migration Wizard. SSIS was a completely different data integration platform when compared to the SQL Server 7.0 originating Data Transformation Services (DTS). DTS contained (like the other 2000-based and older APIs) a COM-based object model implemented in three DLL files: dtspkg.dll, dtspump.dll, and custtask.dll. With SSIS we received a managed object-model like the other 2005-based services.

*In SQL Server 2008 Books Online CTP5 there are several instances mentioning SQL Server 2008 will be the last version of the product to entail the DTS platform. If you have not already done so, plan/migrate your DTS packages to SSIS!

SSAS beyond the API changes was drastically changed. XMLA, Actions, KPIs, perspectives, the concept of measure groups, there were huge changes in the SSAS environment. Like SSIS, SSAS provided a migration wizard for the 2000 database formats to the 2005. In addition, for the first time SSAS supported a project mode in addition to the online mode of SSAS databases. The SSAS project mode is part of a larger difference, which was the introduction of the 2005-based Business Intelligence Development Studio (BIDS). Prior to SQL Server 2005 we had to use a combination of Enterprise Manager, Analysis Manager, Query Analyzer, and the MDX Sample Application for developing SQL Server BI solutions. BIDS consolidated all of the BI development needs into one tool based on the larger Visual Studio shell.

SQL Server Reporting Services (SSRS) was a bit different from the rest of the SQL Server BI components for a good reason; it was originally designed for the SQL Server 2005 infrastructure, it was ‘down ported’ the SQL Server 2000 product in 2004 due to customer demand. SSRS is the outlier because of the previous statement and that it is undergoing the most changes from the SQL Server 2005 to the 2008 product. The Report Server in SSRS 2008 has been completely re-architected. The Report Server in SQL Server 2008 has been consolidated into a single Windows-based service that provides the report manager, web services, and background processing cores. This new Report Server architecture is enabled by leveraging the SQL Server’s networking stack and the HTTP.SYS which is part of the Windows networking subsystem and eliminates the requirement of Internet Information Services (ISS) in the process. SQL Server gurus foresaw this occurring because of the relational engine’s addition of native HTTP endpoints in the 2005 product that leveraged the HTTP.SYS module. Because of this re-architecture effort in SSRS 2008 we do see a few IIS-centric issues in the Upgrade Advisor 2008 release for SSRS.

In conclusion, you will have to incur the ‘heavy lifting’ of the significant differences in the APIs and BI components (SSIS/SSRS/SSAS) regardless of whether you upgrade to the 2005 or the 2008 product. If you were to compare the SQL Server Upgrade Advisor (SSUA) accompanying help you will find a large number of the BI component’s upgrade issues are the same from the 2005 SSUA tool, this is because of the ‘heavy lifting’ I have described in this post that exists between the 2000 & later versions of Microsoft SQL Server. Mainstream support for SQL Server 2000 is coming to an end and so it behooves you to start considering an upgrade to either the 2005 or 2008 version. I’d like to conclude this post with a brief summary table per BI component of the severity of differences between the product versions (with accompanying notes) and thus the more upgrade issues you will face.

Figure 1: SQL Server 2000 to 2005/2008 BI Upgrade Effort Summary Table

Derek Comingore

 

 Business Intelligence: Platform Introduction [9:13m]: Play Now | Play in Popup | Download