Overview

When you want to use Plone to create, edit and manage things like news releases, volunteers, recipes, comic books, music or musicians, you need to create new content types. You'll want forms for entering data, tools to search for it, and pages that summarize it. One of the fantastic things about Plone is that you can do all of this in a way that is easy (if you know what to do), very quick, well documented, and maintainable and sustainable in future releases of Plone. The trick is knowing how to use Archetypes.

And if you know what to do, you can do all of this without any python programming. For the most part, you won't have to work at the command line or even the ZMI. The trick for me, and probably you, is figuring out the easy way. The answer is ArgoUML and ArchGenXML, and this guide. And a video by Sean Kelly.

Without a doubt, the absolute best place to start is by watching the Getting Your Feet Wet With Plone by Sean Kelly. I'll wait here while you get a beer and watch the video. In 20 minutes and 49 seconds you'll see how to install Plone and use ArgoUML to create a time log item type for Plone. You'll come back here all fired up. Go ahead now, I'll wait for you here. I think I'll get a beer too.

Did you come right back here, or did you try to do it yourself? If you tried to build your own product, you may have run into a number of snags. I know I did, like these:

• How do you make the ArgoUML files launch the editor when you double-click them?
• How do you know all the legal tag values and widget types? How do you make them appear as pop-up values within ArgoUML so you don't have to type them?
• How do you make your new type work with smart folders?
• How do you create custom views?
• How do you CSV import your data, geocode it, and display it in a Google map? (Okay, maybe you didn't have that question, but you'd be surprized at how easy it is in Plone.)
  

Sure enough, you can find answers for all of these (and a lot more) in various tutorials, how-to guides, or by searching the newsgroups. And searching the newsgroups. And searching again. And again. That takes lots of time, so here I'm trying to round up all the steps and mis-steps I took in the hopes of saving you time.

We'll work through creating a contact database of volunteers, complete with through the web (TTW) creation of reports like the phone number of volunteers who have canvassed on prior campaigns.

Installing ArgoUML

Assuming you've got Plone installed, the next step is to download and install ArgoUML.

There's a customization file that makes ArgoUML work better with Plone, so you don't have to type the widget types by hand. Scroll to the bottom of the Plone ArchGenXML Issue page to find the most current version of the file.

You usually launch ArgoUML by double-clicking on the argouml.jar file. But to launch it with the new customizations, you have to use the command line. Here's the syntax, more or less:

C:\Program Files\Java\jre1.5.0_06\bin\java.exe
-Dargo.defaultModel=C:\argouml\ArchGenXMLprofile.xmi -jar C:\argouml\argouml.jar

ArgoUML for Windows

Launch ArgoUML and save a project file, even if it's blank. Now, you can make .zargo files automatically launch ArgoUML with the customization file:

Windows: How can I get ArgoUML to load when I double-click on a model file?

    The trick is to do things in two steps. First, open Windows Explorer and find a .zargo file, and then right-click on it. You should get the normal Windows menu, including an Open or Open With option. At this point, give a description for the file like "ArgoUML Model", and tell Windows to use Notepad to open the file. That's not what you want to do ultimately, but it does get Windows to accept the .zargo extension as representing a valid file type.

    Now close, and then reopen Windows Explorer. If you are using a version of Windows prior to XP go to the Explorer menu and select View->Options. For XP you would use Tools->Folder Options. You should get a two (or more) tabbed dialog. Click the "File Types" dialog and then scroll through the list for the Description you gave - in this example "ArgoUML Model". Click to select that file type, and then click the "Edit" button.

    Now, click on the "Open" action and then the "Edit" button there. You will then get a dialog that has a line for entering in the application action that will have a line for using Notepad to open the file. Replace that line with:

"C:\Program Files\Java\jre1.5.0_06\bin\javaw.exe"
"-Dargo.defaultModel=C:\Documents and
Settings\Ivan\Desktop\ArgoUML-0.20\argouml_profile.xmi" -jar
"C:\Documents and Settings\Ivan\Desktop\ArgoUML-0.20\argouml.jar" "%L"

    Substitute your path to your JavaW.exe and ArgoUML.jar files if they aren't located where specified above. Click OK, then click OK, then click OK (three buttons on three different levels of the dialog).

    You can now double-click on your model and have ArgoUML instantly loaded.

(source: http://argouml.tigris.org/faqs/users.html#win_link )

ArgoUML for Mac OS X

I don't (yet) know how to customize ArgoUML for Mac OS, or how to make .zargo files double-clickable on Mac OS.

• On the Mac, you have to switch to a different shell before rehash works. Type tcsh. Also, the bin directory might not be set up. You can use setenv when you're in the tcsh.
• On the Mac, since I couldn't figure out how to load the Tag vocabulary, I had to create them as I went. In the video you just type them in, but in my version of ArgoUML, I had to click on a tag definition, then click 'New Tag Definition', and type in names for each like "widget:type".

Now we're ready to create a model.

Modeling in ArgoUML

This is spelled out pretty well in Sean's video, but a few things weren't obvious or maybe have changed since he created it. For starters, get familiar with these terms. You'll use the UML terms when you're using ArgoUML, but the archetypes terms when you're searching the web or asking a Plone wizard for help.

UML Term	Archetypes term	naming	Description
package	product	camelCase, typically "content"	Collects all the content types, views, etc.
class	content type	UpperCamelCase	There are three basic content types: normal (like a volunteer), folderish, and BTree folderish. Folderish types contain other content types, so a volunteer list might be folderish, or if you thought there would be hundreds or thousands of objects, it'd be BTree folderish.
attribute	field	camelCase	Name, phone number or other content type data. Fields have basic types like string, integer or boolean.
tagged value	property	camelCase	Fields have labels, descriptions, validators, and other properties that describe how they are edited and displayed and so forth. Widgets are properties of fields.
operation	method	camelCase	Used to define computed fields, views. Add a method and set the stereotype on the method.
stereotype	subclass, widget, etc.	UpperCamelCase	To create widgets, portal_tools, large folders or other items, create a class and set the stereotype on the class. Widgets are named TheNameWidget.

• The field types have to be in lowercase. The video shows "String", you should use "string".
  
• Widget names are case-sensitive and in UpperCamelCase.
  
• Basically, most fields are basic types like boolean, integer, fixedpoint, float, string, or text. But there are some interesting ones like lines, image, reference and file.
  

• With ArgoUML, you don't have to export the XMI. The .zargo file seems to include it.
• There are two basic types of items you'll create. Items that stay collected in a folder, like a database. A volunteer list, or maybe a list of books is a good example. This is a class that composites other classes, and it's the type shown in the Wetfeet video, using a filled in diamond in the diagram. A class that aggregates other classes will allow the contained class to be added anywhere on the site. It might be useful for a press release or other type of content. It will be shown as a hollow diamond on the diagram. You can change from composite to aggregate by right-clicking on the arrow.
  

Fields with controlled vocabulary

On many fields, you'll want to give the user a list of things to choose from. This helps ensure they enter a valid value and can make it easier for them. There are several widgets for this: SelectionWidget if they can only pick one, MultiSelectionWidget, PicklistWidget and InAndOutWidget if they can pick more than one. They can appear as radio buttons, check boxes, lists, popups, and so forth.

There are three ways to define the list of options (vocabulary) available to the user. For short lists, just provide the options as a Python list.

Tag	Value
vocabulary	python:['first','second','third']

If you'd like to have more descriptive choices, you can use a Python DisplayList. These descriptive versions will only be shown while editing the form, not when viewing or in Smart Folder tables. Also, note that DisplayList is case sensitive.

Tag	Value
vocabulary	python:DisplayList((('first','1st choice'), ('second', '2nd choice'), ('third', '3rd choice')))

But if the list is longer, or you'd like to be able to change it TTW without having to update your product, use the ATVocabularyManager product. In the UML, you simply add:

Tag	Value
vocabulary:type	ATVocabularyManager
vocabulary:name	vocabListName

I installed ATVocabularyManager, then created vocabularies that matched the vocabListName used in the UML, and this worked perfectly. In fact, if all you want are simple (not hierarchical or sorted) vocabularies, when you install your product, it will create the vocabularies for you. They'll be empty, but at least you know you've got the vocabulary name in Plone to exactly match the one in your product.

Default values

The easiest way is just to add a tagged-value called "default" and fill in the default value. Or, you can select an attribute, switch from the widget tab to the properties tab, and fill in the Initial value field.

Tag	Value
default	Springfield

Changing the field order

This is pretty awful, but select the class (volunteer, for example) and then click on the properties tab. You'll see a list of the attributes (fields). Right click on the field you want to move and you'll see "Move up" and "Move down" options. Repeat until you're happy, or frustrated with ArgoUML.

Repurposing the title field

If you'd like to use something else as the title, like the name of a book that will be typed in, you can repurpose the title field. To do that, add a field named "title" with the following tagged values. Notice that the field name is "title" lowercase, but the accessor is "Title" uppercase. If you'd like to calculate the title field, there are instructions below in Computing the ID and Title fields.

Tag	Value
accessor	Title
required	python:1
widget:label	Comic book name

Smart Folder indexes and metadata

Ultimately, you'll want to be able to filter on some of the fields. To do that, you have to identify those fields to be Smart Folder Indexes. If it's a lines field, you may need to specify that it's a keyword index type. I haven't tried this yet.

tag	value	Comment
index	python:("FieldIndex")	This makes the field visible as an index.
index	python:("FieldIndex:brains")	This makes the field visible as an index, and as metadata.
index	python:(":brains")	This made the field appear in the metadata list, and I could add it to Smart Folder tables. Data on existing items didn't appear right away, until I published them, or edited them again. Presumably reindexing the catalog would work.

Validating field values

There's good coverage of this in The Definitive Guide to Plone, chapter 13. There's also info in the Archetypes quick reference. Here are some that worked for me:

tag	value	Comment
validators	isEmail	Worked
validators	inNumericRange,1,9	Didn't work.

Computed fields

Computed fields are computed at the time you view them, and not stored in the database. First off all, the field type should be "computed", instead of the usual "string". Then give it the following attributes:

tag	value	Comment
widget:type	ComputedWidget	Overall, this all got through ArchGenXML okay, but failed as an installed product.
widget:visible	python:{'view':'visible', 'edit':'hidden'}	It may not matter if it's invisible or hidden in edit mode. In View mode, invisible removes it from the HTML. Hidden leaves it in the HTML, but hides through CSS.
expression	context.getStreetAddress() + ', ' + context.getCity()	Even though my field is "streetAddress", the accessor is "getStreetAddress". It beautifies the case for me.
expression	context.calcFullAddress()	This calls a function you've defined. See the section below for more details.

ArchGenXML

After you save the model and export the XMI, it's time to transform it into a product. Install ArchGenXML on your local computer. It requires Python, but if you've installed Plone on your PC then you're all set. On my Mac, I created source and bin directories and the shadow link as shown in the video.

In Windows, first add Python.exe to your Path environment variable. Right-click My Computer, select properties, select the Advanced tab, click Environment Variables, Edit path in the System Variables. To test this, from the command line, type "python.exe" and you should get the >>> prompt. Type control-z to exit python.

It took several tries to transform the XMI into a product, hence the tips above. Remember, if you're using ArgoUML, you can just use the .zargo file, you don't need to export the XMI.

Next, go to the command line and move to the directory with your Plone products. Your .zargo file or .xmi file should be in there. Test that you can run ArchGenXML with the following:

python.exe \plone-work\archgenxml\archgenxml.py --version

Now you're ready to create your product.

python.exe \plone-work\archgenxml\archgenxml.py volunteerlist.zargo -o volunteerList

When the product was done, install it into a local version of Plone with debug mode turned on. Check the zope.conf file to turn on debug mode.

If Zope is in debug mode, it won't even start if I have problems in your product. Keep tweaking the product and removing complex things until it works. I had to stop and start Zope each time. Eventually, you'll be able to install it and add the new document type as shown in Sean's video.

When I put a "content" class around my other classes, the directory structure of my product matches the typical Archetypes model more closely, but then my product breaks, at least since I added custom widgets. So for now, don't do that. Here's the minimum Archetypes model:

productName/
- Extensions/    # 
- skins/         # contains custom views

Improving your product

For products that are easier to maintain, you should put all of your content types into a "content" package, bundle your model into the product, and include tests. The directory structure will look like this:

productName/
- README.txt     # visible in the Add/Remove Products screen as Product Description
- version.txt    # visible in the Add/Remove Products screen as version number
- content/       # contains the content types
- Extensions/    # 
- model/         #
- - productName.conf     # author and copyright info
- - productName.xmi      # the model for your product
- - generate_source.sh   # a script that regenerates your product
- skins/         # 
- - productName/ # contains custom views, icons and widgets
- tests/         # contains unit tests for your product

This isn't hard to do. For starters, go back to your model and put your classes into a package called "content". That will create the content folder and put them in it.

Next, create a folder called "model" and put the source file for the model into it, along with some information on ownership and licensing. By keeping source with the product you make it easier to patch down the road. Instructions to come, from Nate Aune's tutorial.

Nate Aune of Jazkarta has a great ArchGenXML-ArgoUML tutorial presentation on his website. It picks up where all of this leaves off. His slide that compares UML terms to AT terms (page 16) is very useful. He shows handling photos, internationalization, dynamic vocabularies, how to create relations between different items (artists in groups), computed fields, and provides an introductory look to the behind-the-scenes files that make up the product.

To add a Product Description (and docs) to the product, just put a README.txt file into the main product folder. It can be structured text.

To set the product version number, just edit version.txt in the main product folder. It will automatically increment each time the product is rebuilt.

If you want to create an icon for your content type, put the image into the ProductName/skins/ProductName folder. On the class, add content_icon=ImageName.

Computed Fields, part 2

In the productfolder/product.py, or productfolder/content/product.py, you'll find the following section:

    ##code-section class-header #fill in your manual code here
    ##/code-section class-header

    # Methods

    # Manually created methods

Past your function in under the manually created methods header. You must indent it four spaces, because it is contained in the product class. Here's an example:

    def calcFullAddress(self):
        """
        Put the house#, house suffix, street, apartment, city, state, zip together
        10A Washington St, Apt 3, Chelmsford, MA 01824
        """

        streetPart = str(self.getHouseNumber())+ self.getHouseNumberSuffix()+ ' ' +self.getStreet()
        if self.getApartment() == '':
            streetPart2 = ''
        else:
            streetPart2 = ', ' +self.getApartment()
        cityPart = self.getCity()+ ', ' +self.getState()+ ' ' +self.getPostalCode()

        return streetPart + ', ' + cityPart

Computing the ID and Title fields

There are a couple of common scenarios for this. The first is if you'd like to use something like "Comic book name" instead of Title. For this, just repurpose it as described earlier. The second scenario is computing it by concatenating other fields, like first name + surname to create full name, and using that as the title. Look in the content type file (productName/content/myContentType.py) for your product for the following code:

    _at_rename_after_creation = True
    schema = resident_schema
    ##code-section class-header #fill in your manual code here
    ##/code-section class-header
    # Methods
    # Manually created methods

Now add the code to remove the old title field and substitute your calculation:

    _at_rename_after_creation = True
    schema = resident_schema
    ##code-section class-header #fill in your manual code here
    del schema['title']
    ##/code-section class-header
    # Methods
    # Manually created methods
    def Title(self):
        """
        Put first name, middle initial, last name and suffix together
        Avoid extra spaces from empty fields
        John A. Doe, Jr
        """

        wholeName = self.getFirstName()
        if self.getMiddleInitial() <> '':
            wholeName = wholeName +' '+ self.getMiddleInitial()
        wholeName = wholeName +' '+ self.getLastName()
        if self.getNameSuffix() <> '':
            wholeName = wholeName +', '+ self.getNameSuffix()
    return wholeName 

ID is set automatically, a nice tidy concatenation of the Title/wholeName.

Filtering and displaying the data in Smart Folders

Now I needed two other things: Better columnar views to summarize the data, and searching tools to filter out the records shown in the summary. I got both from Smart Folders.

First, go to Site Setup > Smart Folders and add Volunteers to the search list. Actually, I'll have to change this step later to make it so only certain roles can search for volunteers.

Second, create a Smart Folder. For Criteria, set Item Type = Volunteers. Without that, it won't find anything. Also, use the Edit tab to set the columns in the display table.

Now, the challenge is using the particular volunteer fields for the display and criteria. Mikko Ohtamaa has written a great tutorial on adding fields to Smart Folders. If you've identified fields to be indexed in your model with the index attribute, then you can skip the ZMI portion of Mikko's tutorial and just go the the Site Setup -> Smart Folder Settings. Under the Smart Folder Indexes tab, when you click "All Fields" in the top right, you'll see the fields you identified with FieldIndex and you can make them available to be used as Smart Folder search criteria. Under the Smart Folder Metadata tab, when you click "All fields" in the top right, you'll see the fields you identified with FieldIndex:brains and you can make them available to be used as additional columns in Smart Folder table views.

You can make this happen automatically on installation, but I haven't figured this out yet and it requires programming. (We've avoided it so far, do we want to start now?)

My Smart Folder view was showing the title of matching records, but not the other fields. The problem? It only shows the metadata of fields from published records. As soon as I published them, they showed up.  (Correction, the index wasn't refreshed. Publishing them caused them to show up. Refreshing in the ZMI did too.)

Typical Smart Folders: People who phone bank, people who canvass, people who live in precinct 6, people who worked on Deval's campaign, people in good yard sign locations.

Custom views and widgets

In theory, you can override the default view and create additional ones. To override the default, you create a myProduct.pt file and put it in the skins/myProduct folder. Details to follow.

I created a custom widget, click_email_widget, that makes it when the email address is viewed, it can be clicked. I also created click_map_widget which makes it so when the address field is clicked it is sent to Google maps. Floyd May has a good tutorial called Customizing AT View Templates.

To make your widget install automatically, place click_map_widget.pt into the products/volunteerList/skins/volunteerList folder. Add widget:macro_view=click_map_widget as an attribute of the field.

Tests

More to come...

Other stuff

For CSV import, check out the ArcheCSV product. For CSV export, you'll probably want to create a custom view. Probably something can be done to add a new view for Smart Folders that is similar to the tabular view, but uses commas as separators.

I had trouble using ArcheCSV out of the box because the File Upload button didn't appear. This is due to a conflict with some Kupu settings. There are instructions on resolving it on their support forum. Basically, Kupu added some CSS to hide the button, but it also hides it in ArcheCSV. You have to override the CSS to make it visible in both.

References

Archetypes Developer's Manual -- Required reading. Explanation of Archetypes. Explanation of building a simple InstantMessage Archetypes product, including permissions, startup, classes, custom view, installation, and test. Reference of field and widget types.

ArchGenXML Tutorial -- Required reading. Maps UML to Archetypes vocabulary, covers creating content-types with fields and widgets, methods and actions, and workflows. Includes integration with ATVocabularyManager for multiselection fields, and Relations for complex references between content-types.

Files

I've noted the version number available when this document was written.

ArgoUML -- [Version 0.22]

ArgoUML_profile.XMI -- [Version 0.9] This file adds widget definitions and more to ArgoUML, customizing it for use with ArchGenXML. Read the comments on this page, from the bottom up, to find the most recent version. (I've read this file comes with ArchGenXML but haven't checked that, and it might not be the most recent version.)

ArchGenXML -- [Version 1.5]

ArcheCSV -- [Version 1.0]