Criteria used when Evaluating CMS Systems

Split into summary + key requirements + niceties. In the latter two, a set of desirable properties structured as: property -- goal, rationale, evaluation criteria with scales.

Summary

The summary of the evaluation is the expression of suitability for the preferred site types; for each, scale is [useless/twisted/partly/quite-well/perfect-fit/overkill] + comments on plug-ins or changes necessary to get to that level.

Key requirements

Size -- fairly simple, ssmal but extensible system

I don't need a full-fledged document management with workflow and bells-and-whistles; rather, something that basically allows to put documents into areas. Criteria: unzipped size [KB], plug-in architecture [no/yes], coverage of items below [none/some/most/all].

Platform -- LAMP

Due to the support by the majority of webhosting services. Criteria: object PHP [no/yes], MySQL? [no/yes/plus others]

Install and setup -- simplicity

Quick and automated setup that works in webhosting environment. Criteria: time to go operational [min], web-based config [none/most/all], multiple sites in one CMS instance [no/yes]

Plain text edit -- Textile/Wiki syntax

I explicitly don't want HTML tags written by hand, for two reasons: (1) it is a barrier for average content creators, (2) it may screw up document structure, validity, formatting. Textile or Wiki are really good as the basic tools for text write-up; even I (with all the knowledge of (X)HTML since 1995) prefer them for content creation. Criteria: wiki-like input [no/yes], textile [no/plugin/builtin]

WYSIWYG editing -- cross-platform

Word-like environment is even better for the even more average users, but it must work in both MSIE and Mozilla-based browsers, and preferrably in Opera too. Htmlarea v3 functionality & configurability is ideal benchmark. Criteria: wysiwyg [no/builtin/plugin], configurable [no/yes]

Generated HTML -- valid XHTML

Along the lines of Dogma W4 reasons and goals. Criteria: valid XHTML [no/warnings/clean].

Templating -- clear template and style definitions

It must be simple to define the website's look and feel via the CMS itself, for reasons of personalization, generated code validity and accessibility guidelines. Criteria: user control over generated code structure [none/partly/single-template/variable-templates], website section differentiation possible [no/code-tweaks/yes], snippet templates (content reuse) [no/yes].

Domain-specific applications -- easy to include

Especially for company and community websites, there will be a need to include legacy or new web-applications or dynamic content modules (in-house user management, publications, ...). Criteria: mechanism for app inclusion [none/in-page/plug-in/both].

User management -- delegation of responsibilities

Multiple authors and login-to-edit is a must, able to assign user roles and responsibilities for parts of the website is a desirable feature. Criteria: users and roles [no/just-users/roles], user groups with privileges [no/yes].

Content structuring -- web or magazine?

The CMS should let the user to create a "standard" website with a hierarchy of areas/topics and interlinked pages, not just a toilet-paper blog or "newest on top" news-site. Criteria: hierarchical content [no/yes/multilevel], hierarchical URL [no/simple-setup/complex-rewrite], hierarchical content presentation [no/blog/tweaks/yes].

Localization -- full l10n support

Because we work in non-English environment, the CMS must be easy to cnovert to and use in other languages. Criteria: L10N? and I18N? issues separate in implementation [no/partially/yes].

Niceties

Metadata -- keywords for posts

It is desirable (for SEO and other reasons) to be able to set keywords, descriptions, and other metadata to website parts; at least by sections, ideally per page/post/article. Criteria: can set metadata [no/website/section/page], metadata supported [none/keywords/desc/rating/configurable].

URL features -- search engines don't index query parameters

The URLs? should reflect site structure, and the CMS should provide for URL aliasing. Criteria: link contents [id/id-hierarchy/titles], search-engine friendly URLs? [no/tweaks/mod-rewrite/native].

Archiving -- access to older content

The more history of the content the CMS can accommodate, the better. Criteria: supports archiving (rather than deleting) old content [no/yes/archive-search], content is versioned [no/yes].

Statistics -- enable content re-structuring

If the CMS gathers (passively and actively) visitor stats, we can react. Criteria: usage stats gathered [no/yes], can create polls [no/yes].

Documentation -- admin manuals

Most probably we will need to extend the CMS with our themes, templates, domain-specific applications. Therefore a decent howto/manual for these three topics is desirable. Criteria: templating manual [no/rough/detailed], programming manual [no/rough/detailed].