The road to metadata quality hell is paved with good intentions. Spellbound by the possibilities a complex taxonomy (or just seeming like an interesting mental exercise in its own right), the team gets to work on a detailed taxonomy that could cover all the future needs of sophisticated searching / browsing. You have a problem when the complexity of your taxonomy outpaces functionality the site visitor sees. So if you have a five level deep taxonomy but your site visitors only ever see the first level, then chances are that only the top level is accurately tagged. Since the content contributors won't see any difference / gain, then why would they spend the extra time required?

Match automated functionality to taxonomy complexity

The metadata sweet spot is where the taxonomy complexity matches the automated functionality on your site. So for example the hobbsontech.com site is all tagged to the five stages of CMS migration, and this tagging is used throughout the site (on the home page, as tags with every blog post, etc). So the complexity matches the functionality, and I am compelled to tag correctly. In addition to the functionality, the match must also be in the pain level of bad tagging. So for example I originally structured this site differently so old posts do not have very useful metatadata. Since the pain level is relatively low, I haven't yet cleaned that old metadata (note also that this is a problem with free metadata tags without a large number of contributors). So if you bury some metadata-driven functionality that none of the content contributors care about, then the quality will also be low.

So what to do about this? Carefully consider any additional metadata complexity before adding it to your system. If at all possible, consider a way of increasing the visibility of the metadata through automated functionality on your site. The most direct way would be prominent functionality that the site visitor would see/use (and both the site visitor and content owners / site editors care about). Relying on administrative functionality (for example, ranking groups based on how much content is tagged to their detailed topic without showing this to site visitors) only would be more dangerous since it might encourage people to game the system.

For other related metadata posts, see:

• Taxonomy Mappings: Be Careful When Integrating
• Beware False Precision in Tagging
• Content Re-Use: Roles, Consistency, and Standards

As always on this blog, this post is most relevant for larger sites. Is your site large? Answer four questions to get a quick gauge.

We naturally all have a bias of how we approach improving our web sites (Information Architects through an IA lens, programmers from a technical perspective, etc), but we should all occassionally be looking broadly at what is required for success of your web site. I would also advocate that someone (the product manager) should always be looking broadly at site success.

As always, I advocate starting with the vision of what you are attempting to do. This could be your vision for an upcoming site revamp/migration, or your long-term vision.

After you define your vision, you can then help to combat you and your team's biases by:

1. List out the various factors that will lead to meeting your vision.
2. Evaluating where you are, right now, against those factors.
3. Roughly determining how strong you need to be in each area.
4. Concentrate on the deltas, in particular those that have a wide gap.

So instead of diving into one particular area with a tactical view of your problems (for instance, a deep engagment improving the graphic design of a site), you can concentrate on those that will derive the most value. Also, hopefully this will focus discussions so that you are always going to just enough complexity that will meet your needs.

One way of representing this would be in a graph, highlighting a) the "ultimate" / maximum maturity / most complexity / most impressive / most Amazon-like / fantasy, b) current level, and c) the level required for your vision. See the example below, which highlights those areas where you are meeting/exceeding the capabilities toward your vision (in green) and those that require work (in red). In this example, your design level is exceeding what's required toward your goal (which would be good, but on the other hand might be an area where you could spend less). The widest gap in the example is around setting a shared vision, so perhaps should receive the most attention.

 Most written requirements / use cases are limp, not getting to the essence of the goals (see Where's the juice in your requirements?). Why does this happen? Often, since tough decisions are not being made.

Requirements should be focused and targeted to meet your goals.  Since it is probably more painful to make changes later in the process (for instance, after implementation has started), making the tough decisions needed for focused requirements should be a priority. 

Of course, some readers may argue that traditional written requirements are less effective than a more iterative, generative agile approach. I also like the iterative approach, and in fact it can aid in decision-making (for example, since working code is something that all stakeholders can respond to during the process). That said, sometimes requirements written in advance are needed, for example when selecting a Content Management System.

Some Types of Unfocused Requirements

These are some of the types of requirements documents I've seen that show a lack of decision making:

Shock and Awe

Some requirements documents obscure the real purpose of a system since they are so "impressive". These may inspire comments like "Wow, that requirements document is 100 pages and I barely understand any of it! That must have taken a lot of work! You're so smart!". The main problem here is that if stakeholders cannot understand the requirements, then there cannot be true agreement on them. Although these may impress some, they still probably lack much strength since it's hard to see the point.

Listless

Ambling requirements are also common, and these often result from requirements being gathered rather than analyzed and synthesized. Ambling requirements may contain flashes of important requirements, but are buried along with unimportant requirements or even inconsistent requirements. Listless requirements may also be too detailed in places.

Overly Generic

Some requirements documents read like "car must have a driving wheel and four wheels" rather than "sports car primarily used to race on a racetrack on the weekends". One is certainly a fact but doesn't help in, for example, selecting a car. The second one shows a bit more of the flavor of the true requirement.

One requirements documents can have both of the problems listed above, but they can all be helped by better decision-making.

How to Tell if Your Requirements Are Unfocused and What to Do About It

How to know if you haven't made the decisions you need:

• You resist efforts to prioritize your requirements.
• You are excited about a possible capability, but have not written the related requirements since the relevant decisions haven't been made.
• An outsider reads the requirements and does not understand overall what you are trying to accomplish.

How to make the decisions you need:

1. Define a compelling vision. In other words, big picture what are you trying to accomplish?
2. Make sure the requirements are synthesizing. People do not always know what they want, so everyone's input on the requirements should be synthesized into a coherent whole. A good way to see if the requirements are coherent is to ask someone uninvolved (but aware of the general area you are writing requirements for) understands them quickly.
3. Even though it is hard, prioritize your requirements. Many resist the concept of prioritizing requirements, since a requirement is a "must have". Of course you don't always get everything you want, but, moreover, you want to make sure that your highest-priority requirements are easy and straightforward in your implementation. Perhaps a lower-level requirement could be satisfied with a bit of a hack.
4. Consider the purpose of your requirements. If you are selecting a CMS, then really the only requirements that matter are those that will differentiate one product from another (based on your objectives). If you are handing off development in bulk to another team, then the requirements need to be more exhaustive. The general point is to concentrate on the requirements that matter for your current task at hand.

Your visitor winds up at a piloted section of your site, but the first link they click on goes right back to the old site. That would be a bit like playing a maze game where you suddenly roll into a hole and aren't playing in the game any more (but didn't know the rules or that you were playing a game). How do you deal with that and other linking issues when piloting a site? (Also see background on what/why to conduct pilots before full site migrations or all articles on pilots)

Web Site Visitor Confusion

What are some of the ways that pilot can be confusing for your site visitors? Here are some problems and suggestions for improvement:

Not Knowing Where the Pilot Begins and Ends

One of the keys to deciding what to pilot is choosing whether the site visitors will clearly understand where the pilot starts and ends. So before talking about any technical or implementation details, the selection should just make sense. For instance, if you have a clothing site then a possible piloted site would be the women's clothing store. This might make good sense to your set of users. Note that one of the reasons that you might be undertaking a new site is to consolidate different systems, so you may be piloting a new site that contains content from multiple sites. See the example diagram below, where some content from two subsites is being piloted in one site. Using the clothing example again, this could be where there are currently two separate sites that include all of your clothing, and you are eventually merging to one. You could pilot a women's site that will include content from both of the existing sites. At any rate, the key is that, even before considering the technical implementations, the site user should understand what is piloted and what is part of the old site.

Not Knowing When Leaving the Pilot Site

In addition, there are some ways you can implement the system to make it more obvious for the user whether they are on the piloted or existing site, including:

• Indicating when leaving the piloted site. You may wish to implement a standard intermediate page that the user sees before leaving your piloted site (see #3 in the diagram below). At least that way the user has the opportunity to back up if they want to stay on the piloted site.
• Clearly branding the piloted site. You also want to make sure that visually the piloted site is quite different from the existing site.

To Beta or In-Place Pilot

One question will be whether to just plain replace an existing subsite or to do a Beta that users can optionally try out. One advantage of a Beta is that you can have one entry point (the Beta home page) where you can explain to your users what the Beta is, including what the advantages of the new site will be (see #1 in the diagram below). If launching a subsite in place, then you also need to be ready for deep linking to your site. This can also be an advantage so you can do a live pilot of your rewriting rules, but a disadvantage is not being able to explain the new site as easily.

Unnecessary Linking to "Old" Site

If you blindly cut and paste content, you'll have a whole lot of links going straight back to your old site (like the ball dropping in the whole in the game above). This is because you probably will have hard links to your old site within the content. Some links will need to point to the existing site and others to the new site, depending on if the target content is moved into the pilot as well. If you are doing an in-place pilot instead of a Beta, then the old links should continue to work but you still may prefer to get the links "right" anyway (for example, if you want to use managed links that some CMSes offer).

As an example of unnecessary linking, let's say you have two pages B and C that will be piloted. If you just move the content without changing the links, you'll wind up on the old site when click from page B on the new site. Obviously you want to make sure to link from the piloted B page to the piloted C page (see #2 in the diagram below).

Example Linkages in a Pilot, when using a Beta

Migrations can easily grind to a halt. By methodically launching a subsite as a pilot, and giving time to react to issues, you can partially mitigate against that risk. For more information on pilots, see The Pilot in a Site Redesign or Migration: A How-to Checklist, CMS Proof of Concept, Pilot, Migration, and How to Migrate Content into a CMS. Just to re-iterate one important point, it's not a pilot if you don't plan for a) time for real users to use the piloted subsite, and b) time to fix issues before moving on to subsequent subsites (see more).

How do you select what subsite to pilot?

The pilot needs to launch an actual subsite, but which one? There are two key factors:

• Ensure the subsite will meet the objectives of the pilot
• How easily the subsite can be teased out of the rest of the site

Before diving into some of these details, note that if you have a very large suite of sites that you will be implementing or migration then you may be more likely to launch a site as your pilot (rather than a subsite).

Ensure the subsite meets the objectives of the pilot

You may be tempted to pilot whatever is easy, but that would not really help you much as a pilot since you probably would not learn much of importance. One of the most important things to keep in mind is your pilot needs to test how well your compelling vision for the site will be met, so that you can quickly correct course if your vision is not being met. Including the compelling vision, the subsite you select should meet the objectives of a pilot:

• Test key aspects of your compelling vision
• Test virtually all aspects of the eventual migration:
  • Automated migration
  • Manual migration/editing/entry
  • Deployment (making the site "live")
  • Integration with other systems
  • url rewriting
• Feedback, with the intention of making changes prior to migration
  • From actual web site visitors
  • From the core redesign team (inevitably, the more "real" the system gets, the team will see more issues and uncover more miscommunications)
• Estimate effort levels, which requires:
  • representative content types (again, not just the "easy" ones)
  • not just your super stars doing any manual migration, but a representation of actual users

Many subsites will probably be ruled out by the list. For instance, a subsite that just has one content type and only a few users would not work because a) you won't be getting a good estimate of the effort for moving all content types, and b) you might not get good feedback. Note that you may not be able to accomplish all of these objectives, but by thinking about them you can better choose a site and plan your pilot. On the objective of estimating effort levels, you will also need to establish a method of tracking people's effort (such as using some sort of automated reporting, or even a spreadsheet).

How easily can the subsite be teased out from the rest of the site

Broadly speaking, there are two ways of launching a pilot: a) replacing an existing subsite entirely, or b) as a stand-alone beta site (with a prominent link such as "Try the PuppyWeb Beta!"). Some advantages of replacing a site "in place" are that you are doing a deeper test, with all of your current users on a richer set of functionality. For instance, you can test url rewriting directly to see if your old links will still work (if that's what you are attempting). A stand-alone beta is easier to take down in case of emergency, and also is easier to extricate from the rest of the site. So which approach you take depends on your particular objectives and constraints.

Regardless, you will almost certainly have to think through linkages between the site that you are piloting and the rest of the site. A major factor in deciding on what subsite you pilot may be how easy it is to decouple / couple from the site. One helpful set to defining this is to draw a high-level architectural diagram, highlighting the interrelationships between systems and subsites. These are some of the factors to consider in how the piloted subsite will relate to the existing subsites:

• Internal references between content within the subsite under consideration
• References between your existing site and this subsite
• Other linkages with organization-wide services and reporting, such as site-wide search, site-wide RSS feeds, and site-wide statistics.
• References to the subsite from outside you site entirely (for example, Google references)
• How identifiable the subsite will be to site visitors (will the delineation between piloted and "old" subsites be clear to the visitor?)

If a subsite already "stands alone", then it may be easier to migrate that. If an objective of your new overall site is better integration, then you may be able to highlight that in the pilot (by pulling content from both the "old" stand-alone site and content elsewhere).