The FormQueryPlugin provides a query interface to the TWiki::Contrib::DBCache? module, giving a way to present query results as formatted tables.
When using the plugin each topic is treated as a container for data fields. These fields can be the fields in the form attached to the topic, or can be parsed out of the topic. The database is managed by the TWiki::Contrib::DBCache? module, which must be installed for this plugin to work.
The tutorial section of this topic gives an introduction to formulating and formatting queries using this plugin, and is a good place to start.
Note: Since the last release the following macros have been deprecated. Please get compatible support by loading the appropriate plugins:
Macro | Plugin |
---|---|
CALLMACRO | TWiki:Plugins/MacrosPlugin |
SET | TWiki:Plugins/MacrosPlugin |
ARITH | Still supported in this release, but undocumented and should be replaced by %CALC% from SpreadSheetPlugin |
PROGRESS | Deprecated in favour of GaugePlugin?, though still supported because it doesn't require ay additional software to be installed (progress bars are pure HTML) |
WORKDAYS | Still supported in this release though undocumented and deprecated on the basis that it should be in %CALC% at some point |
tasknotify
script has also been removed and is now shipped separately as the TWiki:Plugins/NotifierAddon.
If the appropriate plugins and contributed code modules are loaded, this release should be fully compatible with previous releases of this plugin.
As well as the form data loaded by the DBCache, the FormQueryPlugin adds the ability to extract embedded tables from topics.
Embedded tables that follow a specific format (defined using the EditTablePlugin) are added to the topic as an array of rows, each of which contains a map keyed on the fields and mapping to the values in the table. This array is added to the topic as a field named the same as the table type. Where there are multiple tables of the same type in a topic, they are concatenated together into a single array. See Settings for more information about how to set up tables.
Topics can also be automatically related to each other by name. The plugin uses something called Topic Relations that allow the creation of two-way relationships between topics. Topic relations from parent topics to child topics are stored in the map of the parent topic using the name of the relation as the field name and an array of references to the maps of child topics. Reverse relations from child to parent topics are also stored. See Settings for more information about how to set up topic relations.
Most of the rest of this documentation describes how to search this database and generate reports.
%FORMQUERY
%FORMQUERY
is the basic query mechanism. A %FORMQUERY
works either on the whole database or, if the query
parameter is given, the results of another query.
Parameter | Description |
---|---|
name | Required, gives the name of the query result for use in %SHOWQUERY or another %FORMQUERY |
search | Required, the search to perform (see Search operators) |
query | Optional, the name of the query to refine. If missing, defaults to the whole database |
extract | Optional, the name of a field in each of the matched maps to flatten out into the returned list. |
moan | Optional, if set to "off" will disable match failed message |
For example,
%FORMQUERY{name=AQuery search="Owner='Main\.Fred'"}% %FORMQUERY{name=BQuery query=AQuery search="Product='Boiled Egg'"}% %FORMQUERY{name=CQuery query=BQuery search="" extract=CookingTimes}% %FORMQUERY{name=DQuery query=CQuery search="Time < '4'"}%will search for all topics with a form field
Owner
set to Main.Fred
, then filter that down to those topics that have Product
set to Boiled Egg
. Then it will extract and flatten out the embedded table CookingTimes
in each matched topic. By "flatten out" we mean that future queries on CQuery
must refer to the fields of the CookingTimes
table, not the fields of the topic, i.e. CQuery
will be an array of all the rows in the embedded table. Finally it will filter down to those rows that have the column Time
< 4 minutes.
Of course there is more than one way to skin a cat. A faster way to formulate the same query would be to say:
%FORMQUERY{name=AQuery search="Owner='Main\.Fred' AND Product='Boiled Egg'" extract=CookingTimes}% %FORMQUERY{name=DQuery query=AQuery search="Time < '4'"}%
FORMQUERY is normally silent i.e. it generates no output unless there is an error, in which case an error description is inserted. This error message can be disabled using the moan
parameter which makes FORMQUERY
totally silent.
Fields can be simple field names or can be more complex, and may even contain embedded searches. See Fields below for more information.
At present there is no way to constrain a search on the contents of a contained table, such as an embedded table or forward relation. However there are usually ways around this; for example, a %FORMQUERY
that uses extract
to flatten all the sub-tables, and then use of the parent relation to complete the search. For example:
%FORMQUERY{name=AQuery search="Product='Boiled Egg'" extract=CookingTimes}% %FORMQUERY{name=DQuery query=AQuery search="Time < '4' AND CookingTimes_of.Owner='Main\.Fred'"}%
attachments.9
* [_N_]
where $n is a number will also get the Nth entry in the array e.g. attachments[9]
* X
will return the sum of the subfield X
of each entry e.g. TaskTable.Effort
will sum the Effort
column in a table called TaskTable
.
* [?search]
will perform the given search over the entries in the array. Always returns an array result, even when there is only one result. For example: attachments[?name='pitcha.gif']
will return an array of all the entries that have their subfield name
set to Sam
.
* [*X]
will get a new array made from field X of each entry in this array. For example attachments[*size]
will get an array of the sizes of each attachment.
Maps
* X
will get the subfield named X
. For example, Product
will get the formfield named Product
.
* X.Y
will get the subfield Y
of the subfield named X
.
* [X] = will get the subfield named =X
(so X[Y] and X.Y are synonymous).
In all cases
* #
means "reset to root". So #.Y
will return the subfield Y
of the Map at the root of the query.
%TABLEFORMAT
%TABLEFORMAT
is like the old Fortran 'format' statement, in that it provides a template for the display of values extracted from a query.
Parameter | Description |
---|---|
name | Required, gives the name of the format for use in %SHOWQUERY or another %FORMQUERY |
format | Required, gives the format of the body rows (or the format to subclass) |
header | Optional, gives the format of the header row |
footer | Optional, gives the string to use to terminate the table |
sort | Optional, defines the sort order |
moan | Optional, if set to "off" disables error output |
The header
, footer
and format
parameters are very similar in application to the parameters used in the standard %SEARCH
TWiki command. For the uninitiated, the header is used to define the first row of the results table:
header="| *Topic* | *Summary* |" header="<table><tr><th>Topic</th><th>Summary</th></tr> |" footer="</table>"
The format
parameter is overloaded so it can also be used to refer to a table format previously defined using %TABLEFORMAT
. This allows you to derive new formats (e.g. with different sort orders) For example,
%TABLEFORMAT{ name=TBLFMT format="|$topic|" sort="Owner,Priority"}% %TABLEFORMAT{ name=SUBFMT format=TBLFMT sort="Priority,Owner" }%Any of the
header
, footer
or sort
parameters in the referenced table format can be overridden by giving new values in the %SHOWQUERY
.
The footer is usually only used for tables generated directly in HTML, in which case it is usually set to footer="</table>"
footer="</table>"
The format is used to define the rows in the body of the table:
format="| $topic | $Summary |"
Any fields in the form in the topic can be output by putting a $ sign before the name of the field. The two special fields $topic
and $form
are available for outputting the name of the topic and the type of the form. You can also refer to fields in related topics using the syntax $field.field
where $field is the name of the relation and field
is a field in the related topic; for example, format="|$Child_of.topic|"
.
It is also possible to expand the contents of tables embedded in the topic, in two ways:
$field[format=FORMAT]
where $field
is the table type, which is named the same as the table topic, and FORMAT
is the name of a format previously defined using %TABLEFORMAT
and suitable for expanding the sub-table.
$field.subfield
where subfield
is a numeric field in the contents of the embedded table (i.e. it contains a number). In this case the result will be the sum of all subfield
values over all the rows in the table.
sort
is used to define the sort order, and is a list of field names (without $'s) separated by commas. You can reverse the sort order for a column by prepending '-' to the field name. For example,
sort="Owner,-Priority"
The default sort method is string comparison. If the field data is numeric, then you can put a '#' in front of the field name to force the sort to be numeric. For example,
sort="Owner,#Priority"
or
sort="Owner,-#Priority"
%TABLEFORMAT
is silent i.e. it produces no output on the page unless there is an error, in which case a descriptive error message is output. This message can be disabled using the moan
parameter.
%SHOWQUERY
%SHOWQUERY
accepts all the same parameters as %TABLEFORMAT
(except name
), and also the following parameters:
Parameter | Description |
---|---|
query | Required, gives the name of the query to show |
row_from | Optional, first row to display |
row_count | Optional, number of rows to display |
%SHOWQUERY{ query=AFORMQUERY format=ATBLFMT }%Note: If you render your results in TWiki table format and limit the rows displayed using row_count (e.g. first 10 from 99 rows), and also use TablePlugin to allow sorting on the headers, then only the 10 displayed rows will be sorted by TablePlugin (and not the full 99 rows in the table).
%TOPICCREATOR
Parameter | Description |
---|---|
relation | Required, name of the relation to use to derive the next topic name |
text | Required, text of the button |
base | Optional, name of base topic to apply relation to. Defaults to the current topic. |
form | Optional, type of the form to include with the new topic. The wikiname of the topic containing the form definition. |
template | Optional, the template topic to use |
The base
is used as the RHS of the relation and a new topic name is derived. This is best shown by an example. If we have the relation:
Dir%A_%B subdir Dir%A
and the base is Dir75
, and topics Dir75_1
through Dir75_54
already exist, the generator will generate topic Dir75_55
. Note that even if there are gaps in the sequence e.g. Dir75_53
doesn't exist, the generator will still create Dir75_55
i.e. one more than the largest existing topic. The behaviour of the generator is undefined if the base does not match the RHS of the relation.
A special hard-coded relation, copy
, is also provided by default that will create a new topic by incrementing a number at the end of the current topic name. For example, if the base is Dir75
and we say
%TOPICCREATOR{relation=copy}%then the next topic created will be
Dir76
(or whatever the next available topic is). If the current topic does not end in a number, it's behaviour is undefined.
The topic generator works by creating the topic name at the time the button is pressed i.e. you can dislay a topic containing a generator, wait a day, hit the button to create the new topic and the name of the new topic created will not conflict with other topics created by other people using the generator in the interim. To do this it uses the autocreate
script installed in the twiki bin directory.
%PROGRESS
- DEPRECATED, use TWiki:Plugins/GaugePlugin instead Parameter | Description |
---|---|
total | Required, total length of the bar in arbitrary units |
actual | Required, where we really are |
target | Optional, where we should expect to be |
If target
is omitted, then it creates a standard two-part progress bar. If all three parameters are used, then it creates bars like this:
If actual < target, e.g. %PROGRESS{total=10 target=5 actual=3}% |
|
||||
If actual > target, e.g. %PROGRESS{total=10 target=5 actual=7}% |
|
MyForm
and may contain a table of attachments.
The first thing we want to do is produce a summary table of all attachments in the web with size > 1024.
The web is represented as a Map (a.k.a hash, or associative array) indexed by the topic name. Within this map each topic is itself represented by a Map, within which there is always a field named attachments
. This field contains a reference to an Array of Maps, one for each attachment. Each attachment Map is indexed by the name of a parameter of the record describing the attachment viz. name
, attr
, comment
, date
, path
, size
, user
, and version
. Pictorially:
* TopicOne? (Map)
* attachments (Array)
1 (Map)
* name=pitcha.gif
* size=2048
1 (Map)
* name=image.jpg
* size=556
* TopicTwo? (Map)
* attachments (Array)
* TopicThree?
etc.
The first thing we want to do is extract a single table of all the attachments in all the topics.
%FORMQUERY{name=Allatts search="" extract="attachments"}%An empty
search
matches all entries, so AttQuery
will now be set to the result of this query i.e. a flat Array of all attachments. We can narrow this down to attachments with size > 1024:
%FORMQUERY{name=Bigatts query=Allatts search="size>'1024'"}%Now we can generate a table of these using
SHOWQUERY
:
%SHOWQUERY{query=Bigatts header="|*Topic*|*Name*|*Size*|" format="|$_up.topic|$name|$size|"}%Note that we have to refer to
$_up.topic
to get the name of the topic because each attachment entry contains a reference to the parent topic's Map ( _up
), but not the topic's name. See the schema in TWiki::Plugins/DBCacheContrib for a list of available fields in each table in the database.
To extract the total size of the big attachments we can:
%SUMFIELD{query=Bigatts field="size"}%Now let's say we want to extract all topics whose form attachment is of type
MyForm
, and then list all the topics referenced in the Othertopic
field of MyForm
.
%FORMQUERY{name=formed search="form='MyForm'"}% %SHOWQUERY{query=formed header="|*Topic*|" format="|$Othertopic|"}%Actiually, this isn't quite enough. What we really want to do is list the
Storable
module even if it is installed. The plugin is much faster if Storable
is installed and enabled.
FQP_ENABLE
setting into the WebPreferences topic for the web.
The remaining settings are defined on a per-web basis.
* FQTABLES
* FQRELATIONS
* HIGHLIGHTMAP
FQRELATIONS
variable in the WebPreferences topic of the web to be processed. This defines a semicolon-separated list of rules for deriving relationships between topics using common portions of names. This is best illustrated by an example. Let's say we set FQRELATIONS
to be:
ReQ%Ax%B SubReq ReQ%A; TiT%An%B TestItem ReQ%AThis describes two rules, the first of which says "Any topic named "ReQ<something>x<something>" is a SubReq of another topic called "ReQ<something>". Single upper case characters with a preceding percent sign are used to define the <something>s. So, in this example,
ReQ1456
has the SubReq
relation to ReQ1456x7
. The reverse relation is also inserted, but appending _of
to the relation name, so ReQ1456x7
has the SubReq_of
relation to ReQ1456
.
Obviously a single topic may have many topics that have the same relation to it (i.e. they are one-to-many), so forward relations are stored as arrays in the maps that represent topics. Reverse relations are many_to_one so they are stored as a simple field in the other topic. Relations are stored as pointers to the maps of the other topic so they can't be printed as if they were simple names. Instead you have to use them to refer to fields within the other topic. For example, in a %SHOWQUERY
showing ReQReQ1456x7
you can't refer simply to $SubReq_of
to get ReQ1456
(the name of the related topic); you have to use $SubReq_of.topic
instead.
The default value of FQRELATIONS
is ReQ%Ax%B SubReq ReQ%A; TiT%An%B TestItem ReQ%A
FQTABLES
variable in the WebPreferences topic of the web. This is a list of the names of the topics that EditTablePlugin uses to find it's table formats. Tables are read in as arrays of maps, with each map keyed on a field name generated by stripping all non-alphabetic characters from the column header. Note that the column header where the table is defined is used, not the column header in the EditTable include= topic.
Two special map entries are added to each table row, topic
and the equivalent of the reverse relation described above, represented by the name of the table with "_of" appended. So if you load a table type "CookingTimes", each row will have topic
set to the name of the topic it was loaded from and CookingTimes_of
set to the map of that topic. This allows you to refer to the fields of the topic that contains a table even after the table has been extracted.
The default value of FQTABLES
is TaskTable
%SHOWQUERY
can be recognised and formatted to your preference.
For example, let's say you want to detect the words "No progress" in a table field and highlight it in bold red text. This is done through a special topic in the web that contains a set of mappings from perl regular expressions that match the words to be recognised and the special formatting. For example, the highlight map might contain the following:
* /(No progress)/ = <font color=red><b>$1</b></font>The /No progress/ defines the perl RE to be matched, and the HTML on the right of the equals defines what to replace it with. The $1 inserts whatever was matched by the RE. The RE must match the whole string in the table field, so the above map will NOT match "No progress with editing"; to match that, you'd have to write:
* /(No progress)(.*)/ = <font color=red><b>$1</b></font>$2Only the first RE in the map matched by the string is expanded. Note that spaces between the '=' and the first non-whitespace character of the RHS are ignored. If you want a space here use
The default name of the map topic is HighlightMap
though you can change this by setting the variable FQHIGHLIGHTMAP
in WebPreferences to the name of the topic to use instead. For example
* Set FQHIGHLIGHTMAP = MyHighlightMap
Just about any formatting can be included on the LHS of the map; for example, you may want to leverage the SmiliesPlugin and translate "pass" to a smiley face and "fail" to a frown.
* /pass/ = :-) * /fail/ = :-(
File: | Description: |
---|---|
lib/TWiki/Plugins/FormQueryPlugin.pm | Main plugin script |
lib/TWiki/Plugins/FormQueryPlugin/Arithmetic.pm | Arithmetic interpreter |
lib/TWiki/Plugins/FormQueryPlugin/build.pl | Build file |
lib/TWiki/Plugins/FormQueryPlugin/ColourMap.pm | Highlight map class |
lib/TWiki/Plugins/FormQueryPlugin/TableFormat.pm | Table formatter |
lib/TWiki/Plugins/FormQueryPlugin/TableDef.pm | Table format definition class |
lib/TWiki/Plugins/FormQueryPlugin/Relation.pm | Topic relation class |
lib/TWiki/Plugins/FormQueryPlugin/ReqDBSupport.pm | Extra features |
lib/TWiki/Plugins/FormQueryPlugin/WebDB.pm | Subclass of Map for a whole web |
lib/TWiki/Plugins/FormQueryPlugin/test.zip | Test::Unit tests |
data/TWiki/FormQueryPlugin.txt | Plugin topic |
bin/autocreate | Topic creation script |
bin/tasknotify | Mailer script |
FormQueryPlugin_installer.pl | Install script |
FormQueryPlugin.zip
in your twiki installation directory.
FormQueryPlugin_installer
to automatically check and install other TWiki modules that this module depends on. You can also do this step manually.
Name | Version | Description |
---|---|---|
TWiki::Contrib::Attrs | >=1.000 | Required. Download and install from the TWiki:Plugins/AttrsContrib code library. |
TWiki::Contrib::DBCache | >=1.001 | Required. Download and install from the TWiki:Plugins/DBCacheContrib code library. |
Time::ParseDate | >=2003.0211 | Required. Available from CPAN. |
Storable | >=2.07 | 2.13 recommended; accelerates cache handling. Available from CPAN |
TWiki::Plugins::MacrosPlugin | >=1.000 | Recommended; used for parameterised includes. Download from TWiki:Plugins/MacrosPlugin repository. |
Plugin Author: | TWiki:Main/CrawfordCurrie | ||||||||||||||||||
Plugin Version: | 1.3 | ||||||||||||||||||
Change History: | 16 Jul 2004: Split into sub-components | ||||||||||||||||||
02 Apr 2004: SimonHardyFrancis? bugfixes incorporated. TimSlidel? patches incoporated for mod_perl, autocreate, and collapsing same-type tables in a single topic | |||||||||||||||||||
05 Feb 2004: Bugfix version | |||||||||||||||||||
11 Jul 2003: Initial version | |||||||||||||||||||
Dependencies: |
| ||||||||||||||||||
Perl Version: | 5.0 | ||||||||||||||||||
Plugin Home: | http://TWiki.org/cgi-bin/view/Plugins/FormQueryPlugin | ||||||||||||||||||
Feedback: | http://TWiki.org/cgi-bin/view/Plugins/FormQueryPluginDev | ||||||||||||||||||
Demo URL: | http://www.owiki.org/FQP/ |
This code is based on an original development of Motorola Inc. and is protected by the following copyrights:
The module is shipped with a perl build file, which should be used for installation and testing. Testing is done using Test::Unit, and is invoked using the 'test' build target. Writing tests is a useful way of feeding back bugs as well. I can't encourage you enough to maintain and extend the tests!
As required for the publication of all extensions to TWiki, this software is published under the terms of the GNU General Public License.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details, published at http://www.gnu.org/copyleft/gpl.html