Good scenarios are difficult to develop (see Shock and Awe or Tightly Focused Requirements and Where's the juice in your requirements?). For a CMS selection they are essential since they will be the backbone for your comparisons between systems. By starting with your target vision and then stepping back to CMS priorities, the list of use cases you will develop, and then finally develop use cases, you will have a better chance of selecting a good CMS for you than starting with the details.

Why Use Cases are Easier for CMS Selections...

Requirements for CMS selections are easier than, for example, full system requirements since no system will be developed based on them. Specifically:

• Only those use cases necessary to differentiate between CMSes is needed.
• For each use case, only the level of detail necessary to differentiate between CMSes is needed.
• Also, you have an opportunity during the selection process to better understand what your needs are (based on concrete demonstrations based on the scenarios, etc.)

Aside from general clarity to help guarantee high quality responses, you also do not want to overwhelm potential bidders (or make them think you don't know what you want or don't know what you are doing).

... And Why They're Harder

But the scenarios for CMS selections more difficult since:

• They require discretion / prioritization (to decide what not to include).
• A lot of long-standing issues (often/usually having nothing to do with the technology) will come to the fore. The temptation will be to gloss over these issues but the start of a CMS selection process is a good time to confront them.
• Ideally you want broad buy-in to the requirements, but at the CMS selection phase things are probably a bit more abstract than stakeholders can easily comprehend.
• There may be a rush to purchase a CMS, so one team is tempted to develop the requirements based on their perspective (perhaps only marketing, or only technical).
• They are often developed without an overall vision for the new site.

How to Develop Use Cases

Given these issues, how should you proceed productively? The answer: starting at the top (the vision) and then drilling down until you have use cases targeted to the selection process:

1. Define vision
2. From the vision, define the CMS priorities
3. List just the titles of the scenarios that support the priorities
4. Write the use cases

Before diving into these steps, one key element is to get buy-in from stakeholders at each steps. By progressively getting more detailed you have the opportunity to get feedback as you proceed -- this way, you do not wind up spending a lot of effort on details before you really know what you're trying to accomplish.

Define Vision

Before getting into details, why are you even doing the migration? Even if it's mostly for back-end reasons (for example, you're current infrastructure can't scale to your imminent load), defining a compelling vision that everyone understands can ensure success of your migration. For large sites in particular, you may want to write an implementation strategy to ensure the vision is possible (along with pros/cons) so that you're not over-hyping the possibilities (for instance, it may be necessary to phase in your vision).

Define CMS Priorities

This particular article is dealing very specifically with the technology, but there's a good chance that success of your implementation will have more to do with non-technical issues than technical ones. That said, the point here is to develop good scenarios for a CMS selection, so the next step toward that goal is to define the CMS priorities (ideally these priorities can also be rank ordered) that support your vision.

List of Scenarios

Next, write out the list of scenarios that support the CMS. By defining your vision and priorities (and, for large sites, developed your implementation strategy), you will now be able to narrow down on what scenarios support those priorities.

Write the use cases

Finally, write out the use cases based on the list that supports the scenarios. Given your priorities, you will have a better idea of what details are important to include within the use cases.

In summary, start with the vision

By getting progressively more detailed, but starting with the question of why you are even attempting to migrate to a new CMS (which will be difficult regardless), you will hopefully get to focused requirements based on your actual needs.

Some related resources on other blogs (from 2006 to the present):

• A Scenario-based Approach to Evaluating CMS Vendors (Tony Byrne of CMS Watch)
• Top 10 mistake when selecting a CMS (James Robertson of Step Two Designs)
• Selecting a CMS: Developing Usage Scenarios (Seth Gottlieb of Content Here)

The road to metadata quality hell is paved with good intentions. Spellbound by the possibilities a complex taxonomy (or just seeming like an interesting mental exercise in its own right), the team gets to work on a detailed taxonomy that could cover all the future needs of sophisticated searching / browsing. You have a problem when the complexity of your taxonomy outpaces functionality the site visitor sees. So if you have a five level deep taxonomy but your site visitors only ever see the first level, then chances are that only the top level is accurately tagged. Since the content contributors won't see any difference / gain, then why would they spend the extra time required?

Match automated functionality to taxonomy complexity

The metadata sweet spot is where the taxonomy complexity matches the automated functionality on your site. So for example the hobbsontech.com site is all tagged to the five stages of CMS migration, and this tagging is used throughout the site (on the home page, as tags with every blog post, etc). So the complexity matches the functionality, and I am compelled to tag correctly. In addition to the functionality, the match must also be in the pain level of bad tagging. So for example I originally structured this site differently so old posts do not have very useful metatadata. Since the pain level is relatively low, I haven't yet cleaned that old metadata (note also that this is a problem with free metadata tags without a large number of contributors). So if you bury some metadata-driven functionality that none of the content contributors care about, then the quality will also be low.

So what to do about this? Carefully consider any additional metadata complexity before adding it to your system. If at all possible, consider a way of increasing the visibility of the metadata through automated functionality on your site. The most direct way would be prominent functionality that the site visitor would see/use (and both the site visitor and content owners / site editors care about). Relying on administrative functionality (for example, ranking groups based on how much content is tagged to their detailed topic without showing this to site visitors) only would be more dangerous since it might encourage people to game the system.

For other related metadata posts, see:

• Taxonomy Mappings: Be Careful When Integrating
• Beware False Precision in Tagging
• Content Re-Use: Roles, Consistency, and Standards

As always on this blog, this post is most relevant for larger sites. Is your site large? Answer four questions to get a quick gauge.

We naturally all have a bias of how we approach improving our web sites (Information Architects through an IA lens, programmers from a technical perspective, etc), but we should all occassionally be looking broadly at what is required for success of your web site. I would also advocate that someone (the product manager) should always be looking broadly at site success.

As always, I advocate starting with the vision of what you are attempting to do. This could be your vision for an upcoming site revamp/migration, or your long-term vision.

After you define your vision, you can then help to combat you and your team's biases by:

1. List out the various factors that will lead to meeting your vision.
2. Evaluating where you are, right now, against those factors.
3. Roughly determining how strong you need to be in each area.
4. Concentrate on the deltas, in particular those that have a wide gap.

So instead of diving into one particular area with a tactical view of your problems (for instance, a deep engagment improving the graphic design of a site), you can concentrate on those that will derive the most value. Also, hopefully this will focus discussions so that you are always going to just enough complexity that will meet your needs.

One way of representing this would be in a graph, highlighting a) the "ultimate" / maximum maturity / most complexity / most impressive / most Amazon-like / fantasy, b) current level, and c) the level required for your vision. See the example below, which highlights those areas where you are meeting/exceeding the capabilities toward your vision (in green) and those that require work (in red). In this example, your design level is exceeding what's required toward your goal (which would be good, but on the other hand might be an area where you could spend less). The widest gap in the example is around setting a shared vision, so perhaps should receive the most attention.

 Most written requirements / use cases are limp, not getting to the essence of the goals (see Where's the juice in your requirements?). Why does this happen? Often, since tough decisions are not being made.

Requirements should be focused and targeted to meet your goals.  Since it is probably more painful to make changes later in the process (for instance, after implementation has started), making the tough decisions needed for focused requirements should be a priority. 

Of course, some readers may argue that traditional written requirements are less effective than a more iterative, generative agile approach. I also like the iterative approach, and in fact it can aid in decision-making (for example, since working code is something that all stakeholders can respond to during the process). That said, sometimes requirements written in advance are needed, for example when selecting a Content Management System.

Some Types of Unfocused Requirements

These are some of the types of requirements documents I've seen that show a lack of decision making:

Shock and Awe

Some requirements documents obscure the real purpose of a system since they are so "impressive". These may inspire comments like "Wow, that requirements document is 100 pages and I barely understand any of it! That must have taken a lot of work! You're so smart!". The main problem here is that if stakeholders cannot understand the requirements, then there cannot be true agreement on them. Although these may impress some, they still probably lack much strength since it's hard to see the point.

Listless

Ambling requirements are also common, and these often result from requirements being gathered rather than analyzed and synthesized. Ambling requirements may contain flashes of important requirements, but are buried along with unimportant requirements or even inconsistent requirements. Listless requirements may also be too detailed in places.

Overly Generic

Some requirements documents read like "car must have a driving wheel and four wheels" rather than "sports car primarily used to race on a racetrack on the weekends". One is certainly a fact but doesn't help in, for example, selecting a car. The second one shows a bit more of the flavor of the true requirement.

One requirements documents can have both of the problems listed above, but they can all be helped by better decision-making.

How to Tell if Your Requirements Are Unfocused and What to Do About It

How to know if you haven't made the decisions you need:

• You resist efforts to prioritize your requirements.
• You are excited about a possible capability, but have not written the related requirements since the relevant decisions haven't been made.
• An outsider reads the requirements and does not understand overall what you are trying to accomplish.

How to make the decisions you need:

1. Define a compelling vision. In other words, big picture what are you trying to accomplish?
2. Make sure the requirements are synthesizing. People do not always know what they want, so everyone's input on the requirements should be synthesized into a coherent whole. A good way to see if the requirements are coherent is to ask someone uninvolved (but aware of the general area you are writing requirements for) understands them quickly.
3. Even though it is hard, prioritize your requirements. Many resist the concept of prioritizing requirements, since a requirement is a "must have". Of course you don't always get everything you want, but, moreover, you want to make sure that your highest-priority requirements are easy and straightforward in your implementation. Perhaps a lower-level requirement could be satisfied with a bit of a hack.
4. Consider the purpose of your requirements. If you are selecting a CMS, then really the only requirements that matter are those that will differentiate one product from another (based on your objectives). If you are handing off development in bulk to another team, then the requirements need to be more exhaustive. The general point is to concentrate on the requirements that matter for your current task at hand.

Your visitor winds up at a piloted section of your site, but the first link they click on goes right back to the old site. That would be a bit like playing a maze game where you suddenly roll into a hole and aren't playing in the game any more (but didn't know the rules or that you were playing a game). How do you deal with that and other linking issues when piloting a site? (Also see background on what/why to conduct pilots before full site migrations or all articles on pilots)

Web Site Visitor Confusion

What are some of the ways that pilot can be confusing for your site visitors? Here are some problems and suggestions for improvement:

Not Knowing Where the Pilot Begins and Ends

One of the keys to deciding what to pilot is choosing whether the site visitors will clearly understand where the pilot starts and ends. So before talking about any technical or implementation details, the selection should just make sense. For instance, if you have a clothing site then a possible piloted site would be the women's clothing store. This might make good sense to your set of users. Note that one of the reasons that you might be undertaking a new site is to consolidate different systems, so you may be piloting a new site that contains content from multiple sites. See the example diagram below, where some content from two subsites is being piloted in one site. Using the clothing example again, this could be where there are currently two separate sites that include all of your clothing, and you are eventually merging to one. You could pilot a women's site that will include content from both of the existing sites. At any rate, the key is that, even before considering the technical implementations, the site user should understand what is piloted and what is part of the old site.

Not Knowing When Leaving the Pilot Site

In addition, there are some ways you can implement the system to make it more obvious for the user whether they are on the piloted or existing site, including:

• Indicating when leaving the piloted site. You may wish to implement a standard intermediate page that the user sees before leaving your piloted site (see #3 in the diagram below). At least that way the user has the opportunity to back up if they want to stay on the piloted site.
• Clearly branding the piloted site. You also want to make sure that visually the piloted site is quite different from the existing site.

To Beta or In-Place Pilot

One question will be whether to just plain replace an existing subsite or to do a Beta that users can optionally try out. One advantage of a Beta is that you can have one entry point (the Beta home page) where you can explain to your users what the Beta is, including what the advantages of the new site will be (see #1 in the diagram below). If launching a subsite in place, then you also need to be ready for deep linking to your site. This can also be an advantage so you can do a live pilot of your rewriting rules, but a disadvantage is not being able to explain the new site as easily.

Unnecessary Linking to "Old" Site

If you blindly cut and paste content, you'll have a whole lot of links going straight back to your old site (like the ball dropping in the whole in the game above). This is because you probably will have hard links to your old site within the content. Some links will need to point to the existing site and others to the new site, depending on if the target content is moved into the pilot as well. If you are doing an in-place pilot instead of a Beta, then the old links should continue to work but you still may prefer to get the links "right" anyway (for example, if you want to use managed links that some CMSes offer).

As an example of unnecessary linking, let's say you have two pages B and C that will be piloted. If you just move the content without changing the links, you'll wind up on the old site when click from page B on the new site. Obviously you want to make sure to link from the piloted B page to the piloted C page (see #2 in the diagram below).

Example Linkages in a Pilot, when using a Beta