Requirements are difficult. I should say good requirements are hard to put together. One thing to look for in requirements, instead of a shopping list, is the juice in requirements. Sure you may need a bag of oranges, but what you're really after is orange juice. Of course, you also need a knife, juicer, pitcher, glasses, and to know how to make orange juice. But really, orange juice on it's own really isn't that interesting a requirement since pretty much anyone can make orange juice. So ideally you want to dig deeper and define more clearly what kind of juice you are attempting (perhaps mint orange juice made at your table?).

Many requirements are written as if each item on a requirements list is a magic pill, where if you just take, for example, one pill of "orange" you'll wind up with juice.

Let's step back and start with a shopping list.

The Shopping List

Many requirements I see look something like a shopping list (not that difficult to put together):

• eggs
• lemons
• butter
• flour

If you were handed this, would you know what's for dinner? It could be:

• sauce
• lemon butter
• bars
• cookies
• muffins

It reminds me of the question you get on the plane: "would you like beef or chicken" when you have no idea how it's prepared. So in the list above, you can be pretty certain that you'll be eating something lemony, but you don't know if it's going to be sweet, savory, an accompaniment, or the main course.

Actually, you don't really know if it's going to be lemony. Perhaps lemons may be needed but you're not sure. Also, you may be using ingredients already in your pantry or refrigerator (perhaps lemon fried chicken?). Or the lemons could just be used to keep your apples from turning brown after you cut them. Or you could only be using the lemons to make lemonade. Someone in the house may have just said "I think we're low on butter" with no specific purpose in mind.

The Objective

What if you were handed a sheet that said "We're having your favorite tonight: lemon pancakes! To make the pancakes we need sweet butter, large organic eggs, those small lemons at the back of the store, and flour"? That's a bit more motivating, and something that everyone can understand and line up behind ("Junior, if you don't run over and get lemons we're not having lemon pancakes tonight").

Unstated Requirements, Important Details, Technique and Your Skills

So you get home, and find that you're the one making dinner. All you have is the objective (lemon pancakes) and the ingredients from the shopping list. No one gave you a recipe. And frankly you don't know your way around this kitchen. You look on the internet and find a recipe for lemon pancakes, but it calls for refined sugar which you're avoiding. Come to think of it you just remembered that your daughter is pushing you all to eat organic, and you got regular flour. Thinking that perhaps you can get away with the non-organic flour, you look around, and then see a recipe that uses honey rather than refined sugar. It looks like this recipe is actually harder to make, since they say honey browns really quickly. You vaguely remember that the stove runs hot, or is it cold? I'd probably burn regular pancakes! A sifter? Where's a sifter? Do we have a sifter? We have baking soda but not powder.

Let's just make ramen noodles.

How did this happen? The shopping list was certainly correct: you did need lemons, butter, eggs, and flour. But even the list wasn't specific enough (for example, you needed organic flour). Also, there was a lot of implied/assumed stuff that was actually really very important to making the dinner. This included technique, unstated requirements, and your skills.

Does this kind of thing happen in large systems development? Yes. Someone thinks you should make lemon pancakes, no one knows that's what you're making, and you wind up with ramen noodles. The objective (compelling vision) should be clear to everyone, and the important details should be worked out. In large systems development, a useful way to do this is use cases since they are more easily understood by a wider group of people. Also, it helps pull things together to illustrate the point of the requirements. Is there juice in your requirements?

A previous blog post covered some of the types of metrics to track progress in a migration. This blog post explores ensuring that quality is acceptable during the migration.

A Naturally Iterative Process

Whether you have a high degree of human intervention or not, you will be following a iterative process like the one illustrated below. Basically, you'll have a stream of content/sites/functionality/etc that you are migrating. As you migrate, you are checking the progress by some means or another. The worst case would be the internal team not checking at all but then getting feedback from actual site visitors after launching a part of your site. Even in that case, you will find mistakes and know what needs to be fixed in the next round.

Let's look at these steps. For illustration in this blog post, the concentration will be on quality of content, although of course the drive to progress on other migration metrics needs to be high quality as well.

The Stream

You have a choice in how to phase your project, and you should ground your phasing in moving toward your compelling site vision. By keeping the above process in mind, you can further optimize what you start migrating. Probably the largest issue you want to avoid is re-processing any content twice (that's the "re-apply as needed" box in the graph above), which is covered below in Short and Focused Iterations.

Apply Rules

The next step is to actually migrate content. Since you want this process to be as streamlined as possible, I list this simply as Apply Rules in the process. The key about rules is that, if defined correctly, you can have more consistent treatment (and results) of your content.

In an entirely manual process, you would hopefully have some sort of written checklist/process for those using the tools to migrate the content. The process might be something like: "a) indicate in site inventory that you are working on the page, b) turn into XHTML, c) add descriptive image descriptions, etc". At any rate, some rules would be followed during the actual migration.

In a more automated environment, the rules would be embedded in the scripts/tools used for migration. Note that not all scripts are created the same, and you should challenge your system integrator to creating scripts or configuring tools that can do things like scrape pages that might not be totally regular. Some questions to ask your system integrator include: a) will you be scraping pages, b) will links/images/documents be "managed" when migrated, c) how will references (links, documents, images, etc) be handled such they are consistent, and d) how is acceptable quality determined (see the next section for more)?

Find Mistakes

If you are applying rules to your migration, why a separate step in the process to check your work? One reason is simply that mistakes are found pretty much without anyone explicitly doing anything, so it's important to know what to do when mistakes are found (the following steps in the process). For example, you may have done a careful job migrating some content, only to have site visitors find issues with the content after it has already launched. Aside from the situation where mistakes are found "too late", some reasons to explicitly search for mistakes are: a) a different "pair of eyes" (whether a person or a separate script) can help find issues not found in the first go-around, b) it's often easier or lower-risk to check for errors (if automated) than to ensure correct migration, c) correcting mistakes earlier rather than later is more efficient (especially to avoid doing anything over), and d) you may not have the resources / backing to do specific testing later.

What to check when looking for mistakes? Although simply having people eyeball the content, a checklist would be helpful to make sure key elements are being checked. Note that the checks ideally cover what people, automated scripts, and the system has generated. Some example items to check: completeness (for instance, when checking a page as opposed to a piece of content, making sure that all parts of the page are fully formed vs. "no records found" errors), your standards / style guide, consistent layout (old, large tables not blowing out pages for example), persistent issues, and unnecessary code stripped out and only your styles used (so that it's not just about how the content looks now, but that when you change your CSS the look will change as well). Note that some of these tests are checking the content in various contexts and not just the content alone (since it may be far easier to discover content problems by looking at the pages that are to pull the content).

For a large migration, you probably will not be able to manually check every page. For any page, you have the choice of whether and how to check for mistakes, which broadly speaking fits into three possibilities:

• Manually checking
• Automatically checking
• Not checking

The decision on which of the three paths to take can be done by a variety of ways, including: by content type, by size of page/content, complexity of page, and importance to the compelling vision for your migration. If blocks of the content is highly regular, and manual checking seems to indicate that the migration works well for this content type, then automatic checks probably makes sense. For archival information formatting and small irregularities may not matter, not even checking it may be acceptable.

Change Rules

Whether it is checklists or automated scripts, the rules should be changed so that either the issue doesn't come up again (preferable) or at least there is an easy (hopefully automated) way to check. An unacceptable process is that mistakes from automated scripts are simply siphoned off into a bucket of items that need to be dealt with manually. Not only is this inefficient, but if you let that bucket of problem pages grow, you could inadvertently be blinding yourself to a major underlying issue that cannot be dealt with one by one.

Re-Apply As Needed

This is a key point. You could appear to be migrating along swimmingly, and then find an issue fairly late in the process. Do you go back and re-apply the new rules to the old content? Obviously you want to avoid either the impact or occurrence in the first place. To limit the impact, you could implement things in a way that it's easy to re-run automated scripts on content for tweaks. To limit the occurrence, short and focused iterations might help.

Short and Focused Iterations

There is no way to anticipate the issues you will encounter during the migration, but, especially since you want to avoid having to repeat any steps (a la "Re-Apply As Needed" above), short and focused iterations can help. By short and focused I mean:

• Trying to define what is acceptable quality up front
• Migrating a small amount of easy as well as very difficult content, representing as much as possible the full breadth of the type of content that will be moved
• Implement as much functionality (see other examples) as possible, to better expose issues earlier rather than later (see Why It's Difficult to Migrate)
• Check as thoroughly as possible as early as possible

The above list of suggestions concentrates on initial steps of the migration. Further into the migration, a key point is to make sure you understand the issues (meaning, you know what happened and how to fix it) you are encountering as early as possible, attempting to make sure that there are no deeper systemic issues.

So you have a bright shiny CMS configured, and you're ready for your internal users to begin hands-on CMS work. Do you just let the CMS vendor give their standard training and leave it at that? No. Training on the generic mechanics of the tool is important, but your site/organization is unique and needs specific training. Also, the out of the box training may cover items that actually are not important to your organization, adding unnecessary bulk to the training.

So, what do you need to train on?

This may sound obvious, but your training should focus on what your team needs to know in order to manage their content and/or sites. For example, if you look at Team Experience, Your Tools, and Your Web Standards, then you should train at the intersection of these items: training only on what the team needs to know, based on your use cases. For example, if a key group of users is going to be publishing content but not managing sites, navigation, etc, then this user group should ideally be trained only on how to do that task. Additionally, hopefully the tools and team experience further clarify what needs to be taught. After all, if your tool is enforcing some standards, then, aside from explaining why the enforcement is happening, training does not need to dive into those details. While ramping up for migration, you may consider training just on what the users need to do during the migration, which may be less than what they need to know for ongoing maintenance (this way, you don't confuse everyone with details they'll probably forget anyway, and then you can train on more specifics when they're about to use the new knowledge).

The following are some of the elements that should be considered for training.

The Mechanics of the CMS

It's true that the CMS users need to be able use your CMS, so they do have to be trained on these details. Assuming you've defined major use cases for your site (right?), you could concentrate your training on the use cases for the users you are training. If possible, hand out a cheat sheet so that your users will be able to conduct their business once they forget all the details (and hopefully they don't have to resort to the manual every time). At any rate, the user should only be taught what they need to complete their job, and in the context of the tasks that they complete. Also, this presupposes that the CMS has been configured in a way that facilitates the key use cases.

Writing for the Web and Basic HTML

Your users may already know basic HTML and how to write for the web, but often one of the objectives of a CMS migration is to decentralize publishing so you may have new folks. Yes, of course your new CMS has a phenomenal WYSIWYG editor (or so it appears -- I've never met a web-based editor I didn't end up not liking, hence me writing this blog post with a "fat client"), and hopefully you've been able to embed many of your standards in the editor (like the users can select styles but not fonts). That said, there are still some elements of good HTML that everyone should understand and be trained on. Also, if your CMS copies and pastes from Word, then you'll definitely need specific training on that (for instance, to indicate what to expect in the cutting and pasting). At any rate, depending on your users, the tool and how it was configured, and what you are accomplishing, you may need to teach some basics about writing for the web and basic HTML.

Your Web Site Standards

The tool will not be able to enforce all of your web site standards (although ideally standards are enforced as deeply in the system as possible). There will probably be some contention about the need for standards, so some background on that should be explained in the training. Aside from this important aspect, it's important to focus on the standards that the tool cannot enforce. Some will be items like writing for the web, covered above, but you may have other standards that need further training.

Tagging

Chances are, one of the reasons you are using a CMS is to re-use content. Also, sloppy tagging has such a major impact (and is so easy to happen) that training here is essential. The items that need to be covered are:

• Why tagging is important. An explanation (and hopefully "live" examples) of how tagging impacts where content appears, and how it affects search.
• How to use the tool to tag. Hopefully the tool doesn't hinder tagging.
• Rules (and rules of thumb) of how to tag. Do you tag to force content on certain pages, or only because it's actually about the topic or other attribute? How many values should be selected? What are the rules around the different values?

Of course, this depends on who is doing the tagging, and if you are going to have dedicated people tagging or if you will be doing automated tagging.

Workflow and Publishing Process

Some of the team may still be used to directly ftping files (or even using a lightweight CMS at home), but at any rate systems work (or are configured) differently so this is probably an essential part of the training. Obviously the workflow (clearly stating what the role is of everyone being trained) needs to be covered, but also items like caching need to be understood as these can mean frequent calls to the helpdesk if everyone doesn't know how it works.

Site-specific or Group-specific Information

This last point is easy to overlook but important for large complexes of sites. Different sites driven off the same platform might have slightly different flavors, requiring particular rules for each site. For example, the home page of major sites might be largely custom-managed, and hence needs consistency at the site or group level. Large sites should consider having their own style guide, and train one-on-one with new staff to the group in order to make sure the organization-wide training meshes with the realities of the group.

Before undertaking your large site migration, you should define how you will track progress. As mentioned in a previous post, you'll want to be moving toward a compelling vision of your site. As such, you probably do not want to be tracking only the page count that's been migrated (aside from inaccurately tracking your progress, your overall quality may be better by reducing as much content as possible). One possible approach is to track a variety of dimensions, which would result in status graphs like this:

In this example, 90% of the pages are in the system, 50% of the templates are done, 30% of the major functionality is complete, 20% of the sites are complete, and 10% of the integration with other key systems is complete. This is probably a very dangerous situation, threatening major reworking of the pages that have already been migrated (since, for example, once diving into the sites and functionality you may realize you need different metadata). It probably doesn't make sense to have the percentage complete of each of these progressing in lockstep, but by tracking things in this way you can highlight issues. Also, you more clearly see how much work you have left. In the case above, if you were only looking at the percentage of pages complete, you would think you were almost done. That said, the graph above clearly indicates that is not the case.

So what are some of the dimensions you will want to track?

Pages or Content Items

Depending on your site and objectives, you may want to be counting pages or content items. In a site with significant content re-use, tracking content items (along with sites/templates) will make more sense. Remember again that objective isn't to ram a lot of content in place, but only meaningful data (see my previous Web Diet blog post). In addition to pages or content counts, another item to potentially track is content types.

Sites

If you're only migrating one site, then obviously this isn't a metric to track during migration. That said, if you are migrating multiple sites that are driven off of the same platform, then it probably does make sense. Tracking when sites are actually launched is probably the most useful (rather than when they are "ready" to be launched), so that you are actually confirming that it works out in the wild (search traffic comes to your site, you don't hear about major issues from users, etc).

Templates

The look and feel usually receives a lot of attention, so tracking when templates are implemented would be compelling to a wide range of stakeholders. In addition to the look and feel, the template's functionality should be complete before a template is declared Done. For example, if there are auto-pulls for a template, then the template is not complete unless the auto-pulls are complete.

Integration

Large sites will usually integrate with a variety of existing systems, especially internal systems but also external systems. These may be largely independent of many of the other factors you are tracking, except for data/content format and common taxonomies. Also, since working with other teams can take a long lead time, tracking these and putting the focus on early collaboration is important.

Functions

Your site objectives might include new functionality, and, depending on the complexity, these can take a while to implement (including the need for iteration as you explore what really needs to be implemented). Functionality could include things like quizzes, maps, contact form, calculators, data manipulation tools, special search tools, and conference/event management.

Languages

If your site will have multiple languages, then you may want to be tracking certain functionality (it's not just about being Unicode-enabled!) but also tracking when different languages are fully complete. This will include translations, and you'll need to define how complete different languages need to be.

Channels

Sites are expected to share content in multiple streams / channels, and this may be an important aspect of what you are trying to accomplish in the new site. Example channels could be, in addition to your web site(s): email alerts, Newsletters, APIs, RSS, and other sites (like Swivel for data). Getting the channels right has similarities with the integration points, but in general channels are more generic. Aside from making sure the data is compatible with multiple channels, you'll need to have your database layer well implemented and also define the mechanisms that the content can be accessed (which will include defining the url structure for accessing these different tools).

These are just some examples of metrics to track during a migration, but hopefully give concrete enough cases to help think through what makes sense for your site.

I hope to cover in a future blog post how to actually go about confirming how far you are on each of the metrics you track, since self-reporting is not desirable. In addition, by defining how you will measure in advance, you may avoid issues in the first place since the system will be designed to satisfy those metrics.

As always, I would appreciate any comments.

Have you had a large site migration discussion where someone said something like this?

We'll just start with the easy stuff first

I would argue that, perhaps after a very initial laugh test of an easy use case, in general you should jump straight to testing/piloting/migrating the compelling vision (see description) of your web site (and of course also in your CMS selection as well). Chances are, this means iterating on some tough details rather than plowing through lots of easy work .

Why Easy first?

So what are some of the reasons that project teams give for migrating in easy sites first?

• Need to show results (project's already over budget, late, etc)
• Need to get momentum (if we don't get people bought into this soon, this project's never going to happen)
• Learn from easy cases (let's take baby steps)

Also, another reason is probably that everyone is just a bit overwhelmed by the volume of a large site migration, although integration, functionality, automatic pulls, multi-site management, or multilingual issues may actually be bigger issues.

Issues with Easy First

Efficiency

 Ideally, you only want to touch content once during the migration. If you concentrate on just getting in the simple pages / content / sites, then you may overlook some transformation of the pages necessary for more sophisticated functionality. This might include tagging needs, or structuring pages and content in a way that enables your desired functionality. In general it would be less efficient to come back to the content to correct those issues after already touching each piece of content once.

Buy-In

If you start by implementing easy but low-impact sites or content, then stakeholders may become frustrated that they're hearing trumpeting about progress but no changes that matter to them. If you have got everyone bought into a compelling vision, then to facilitate buy-in you should be trying to show this vision rather than hand-waving.

Wrong Focus, Missing the Vision

If you are concentrating on moving pages, then you'll move in pages. If you are tracking progress with metrics based on number of pages, then the focus will be even stronger. So you may not be looking at functionality, metadata, consistency, etc, that might be key to your vision. Although it may be possible to "undo" mistakes made, it may be politically extremely difficult to go back to retrofit the original vision in the site if everyone's focus has been on getting content into the system.

Metadata Quality

You are probably expecting the CMS to do some automated pulls of content, that will require accurate and complete metadata. If you do not implement the functionality that will expose the tagging (either directly by displaying it on the site or implicitly by pushing it to particular pages), then it will be very easy to not worry about whether the metadata is good when it is being migrated (by hand or automatically). Even more troublesome, you may discover that your metadata model was not sufficient, which would be better to discover earlier rather than later.

Train Wrecks at "End" of Project

By doing the difficult functionality earlier in the process, you're less likely to have the project come to a halt toward the projected end of the project. Running into issues earlier gives you more time to react to them, and also you have invested more effort already. An iterative approach, toward your vision for the site, would be more effective than putting off the more difficult items until later.

Tracking Error

If you're starting with simple stuff first, it's easy to make unrealistic extrapolations. For example, let's say you have 500,000 pages to migrate, and you have migrated 300,000 pages already. You may be tempted to extrapolate with something like the graph at the right. Since it took 3 months to get 300,000 done, it will only take 2 more months to get the last 200,000, right? As discussed above, you may actually be obscuring major issues, which may grind the process to a halt and actually require going back and touching the content that was already migrated.

How to Phase with the Compelling Vision in Mind

Let's say your web site vision is to be the "Authoritative source of mental illness information available via a variety of channels (web, RSS, API), different types (raw data, analysis), and selectable by topic pulling from a variety of back-end systems."

If you are going from the authoritative source you envision, you might start by: a) designing the metadata carefully, b) define how automatic pulls will work, and c) defining standards around the content so that it can be used effectively by multiple channels. Then you might migrate in content that is re-used a lot (or first just pull from one of the back-end systems that's already in place). You might also start by negotiating with the owners of the various back-end systems to determine when they can all meet the same standards for metadata, content-reuse, etc. For the functionality of the site, it probably would make sense to implement a little bit of support for all the functionality you were after, so you might: ensure your system drives a bit of web, RSS, API etc for a single content type first. In addition, it would make sense to start with strong content types first rather than allowing a generic content type for most content.

The near-term result may be a small amount of content, but the highest-value (as the authoritative source) information available on some key pages and via RSS. This would be in contrast to a rush to implement "easy" / high volume, with lots of pages/content but little value.