This module supports structured queries over a database built on the fly from the forms in TWiki topics. It does not support any tags, as it is provided as a service for other plugins that want to treat a TWiki web as a simple database; for example, the TWiki:Plugins/FormQueryPlugin, which supports the display of query results.
The plugin encapsulates code that was formerly in the "engine room" of the FormQueryPlugin. It has been abstracted out in the belief that it will be useful to anyone who wants to do simple search operations from a plugin.
Each topic in the web automatically gets a number of standard fields, generated by reading data and metadata (see TWikiMetaData?) from the topic:
name
- name of the topic
parent
- name of parent topic
web
- name of the topic web
_up
- reference to the Map of the parent topic, if it exists
attachments
- array of Maps, each of which contains:
_up
- reference to the Map for the topic
name
- attachment name
attr
- e.g hidden
comment
- attachment comment
path
- client path used to upload attachment
size
- size in Kb
user
- who uploaded the attachment
version
- e.g. 1.3
info
- Map containing:
_up
- reference to the Map for the topic
author
- most recent author
date
- date of last change
format
- topic format version
version
- topic version number
moved
- Map containing:
_up
- reference to the Map for the topic
by
- who moved it
date
- when they moved it
from
- where they moved it from
to
- where they moved it to
form
- form name
MyForm
); a Map containing:
_up
- reference to the Map for the topic
text
- raw text of the topic
Reminder: The sub-Maps created for info
, form name, moved
, and each row in attachments
also have a reference back to the topic Map, called _up
.
Other fields may be added by subclasses. Refer to the documentation for clients of DBCacheContrib for more details.
_DBCache
, albeit the file name can be overridden in a descendant. If any topic changes in the web this cache is automatically rebuilt. The cache file can be deleted at any point with no ill effects.
The module is shipped with a perl build file, which should be used for installation and testing. Testing is done using Test::Unit, and is invoked using the 'test' build target. Writing tests is a useful way of feeding back bugs as well. I can't encourage you enough to maintain and extend the tests!
TWiki::Contrib::DBCache
class. The following POD documentation describes the methods of this class and the various other classes provided by the plugin..
General purpose cache that treats TWiki topics as hashes. Useful for rapid read and search of the database. Only works on one web.
Typical usage:
use TWiki::Contrib::DBCache; $db = new TWiki::Contrib::DBCache( $web ); # always done $db->load(); # may be always done, or only on demand when a tag is parsed that needs it # the DB is a hash of topics keyed on their name foreach my $topic ($db->getKeys()) { my $attachments = $db->get($topic)->get("attachments"); # attachments is an array foreach my $val ($attachments->getValues()) { my $aname = $attachments->get("name"); my $acomment = $attachments->get("comment"); my $adate = $attachments->get("date"); ... } }As topics are loaded, the readTopicLine method gives subclasses an opportunity to apply special processing to indivual lines, for example to extract special syntax such as %ACTION lines, or embedded tables in the text. See FormQueryPlugin for an example of this.
new($dataDir, $web)
$dataDir
- location of cache file
$web
- name of web to create the object for.
$topics
- perl array of topic names that have just been loaded (or reloaded)
Load the web into the database. Returns a string containing 3 numbers that give the number of topics read from the cache, the number read from file, and the number of previously cached topics that have been removed.
$archive
- the TWiki::Contrib::Archive being written to
$archive
- the TWiki::Contrib::Archive being read from
man perlre
for help). An embedded search may also be applied to a field value. In addition, the left-hand and right-hand sides of a search expression may also by of the form $CALC(
text )
, where text is some text string surrounded by quotes which is evaluated by the SpreadSheetPlugin. The special operator $T
interprets its single argument as a field expression. (The quotes need to be escaped if within a string.)
Warning single and double quotes are not allowed in values!
The following operators are available:
Operator | LHS | RHS | Meaning |
---|---|---|---|
l = r | field name or calc | regular expression string or field name or calc | Value exactly matches this regular expression. The expression must match the whole string. |
l != r | field name or calc | regular expression string or field name or calc | Field is not this RE. Inverse of = |
l =~ r | field name or calc | regular expression string or field name or calc | Value contains this regular expression i.e. the RE is found somewhere in the field value. |
l < r | field name or calc | integer (string containing an integer e.g '4') | field value is < integer |
l > r | field name or calc | integer | field value is > integer |
l >= r | field name or calc | integer | field value is >= integer |
l <= r | field name or calc | integer | field value is <= integer |
l [? r ] | field name | search expression | search applied to field value |
A search string consisting of text somestring
without any operator is interpreted as the string text=~'somestring'
.
A search string consisting of a single =%CALC% expression is interpreted as a test of the result of the expression.
Searches may be combined by the following logical operators:
Operator | LHS | RHS | Meaning |
---|---|---|---|
! | none | expr | Boolean NOT |
AND | expr | expr | Boolean AND |
OR | expr | expr | Boolean OR |
() | N/A | N/A | Bracketed subexpression |
A search object implements the "matches" method as its general contract with the rest of the world.
$db = new TWiki::Contrib::DBCache( $web ); # always done $db->load(); my $search = new TWiki::Contrib::Search("date EARLIER_THAN '1st January 2000'"); foreach my $topic ($db->getKeys()) { my $attachments = $topic->get("attachments"); foreach my $val ($attachments->getValues()) { if ($search->matches($val)) { print $val->get("name") . "\n"; } } }
create($string)
$string
- string containing an expression to parse
matches($object, $case)
-> boolean $object
- object to test; must implement get
$case
- boolean flag; the match is casesensitive if true
$object
can be any object that provides
the method "get" that returns a value given a string key.
toString()
-> string
Object that handles a file/time tuple for use in Storable and
TWiki::Contrib::Archive
.
new($file)
$file
- filename
uptodate()
-> boolean
toString()
-> string
write()
read()
Generic array object. This is required because perl arrays are not objects, and cannot be subclassed e.g. for serialisation. To avoid lots of horrid code to handle special cases of the different perl data structures, we use this array object instead.
new()
add($object)
$object
any perl data type
find($object)
-> integer
remove($index)
$index
- integer index
get($key, $root)
-> datum $k
- key
get("9", $r)
where $n is a number will get the 9th entry in the array
get("[9]", $r)
will also get the 9th entry
get(".X", $r)
will return the sum of the subfield X
of each entry
get("[?search]", $r)
will perform the given search over the entries in the array. Always returns an array result, even when there is only one result. For example: [?name='Sam']
will return an array of all the entries that have their subfield name
set to Sam
.
#
means "reset to root". So get("#[3]", $r)
will return the 4th entry of $r (assuming $r is an array!).
get("[*X]", $r)
will get a new array made from subfield X of each entry in this array.
Where the result of a subfield expansion is another object (a Map or an Array) then further subfield expansions can be used. For example,
get("parent.UserTable[?SubTopic='ThisTopic'].UserName", $web);
See also TWiki::Contrib::Map
for syntax that applies to maps.
size()
-> integer
sum($field)
-> number $field
- name of a field in the class of objects stored by this array
search($search, $case)
-> search result $search
TWiki::Contrib::Search object to use in the search
$case
- boolean flag; the match is casesensitive if true
TWiki::Contrib::Array
of matching entries.
getValues()
-> perl array
Get a "perl" array of the values in the array, suitable for use with foreach
toString($limit, $level, $strung)
-> string $limit
- recursion limit for expansion of elements
$level
- currentl recursion level
$archive
- the TWiki::Contrib::Archive being written to
$archive
- the TWiki::Contrib::Archive being read from
new($string)
fastget($k)
-> datum $k
- key
get($k, $root)
-> datum $k
- key
$root
what # refers to
$k
; return undef if not set.
Subfield syntax
get("X",$r)
will get the subfield named X
.
get("X.Y",$r)
will get the subfield Y
of the subfield named X
.
#
means "reset to root". So get("#.Y", $r)
will return the subfield Y
of $r (assuming $r is a map!).
Where the result of a subfield expansion is another object (a Map or an Array) then further subfield expansions can be used. For example,
get("UserTable[0].Surname", $web);
See also TWiki::Contrib::Array
for syntax that applies to arrays.
set($k, $v)
$k
- key
$v
- value
size()
-> integer
remove($index)
-> old value $index
- integer index
getKeys()
-> perl array
Get a "perl" array of the keys in the map, suitable for use with foreach
getValues()
-> perl array
Get a "perl" array of the values in the Map, suitable for use with foreach
search($search, $case)
-> search result $search
- TWiki::Contrib::Search
object to use in the search
$case
- boolean flag; the match is casesensitive if true
TWiki::Contrib::Array
of matching keys.
toString($limit, $level, $strung)
-> string $limit
- recursion limit for expansion of elements
$level
- currentl recursion level
$archive
- the TWiki::Contrib::Archive being written to
$archive
- the TWiki::Contrib::Archive being read from
This module is only used if Storable isn't available. Storable is much faster, because it is implemented in C.
new($file, $rw)
$file
- archive file path
$rw
- mode "r" or "w"
close()
writeByte($b)
$b
- byte to write
writeString($s)
$s
- string to write
writeInt($i)
$i
integer to write
writeObject()
read($archive)
and write($archive)
, or may be undef or a string. No other types
are supported.
readByte()
-> byte
readString()
-> string
readInt()
-> integer
readObject()
-> object
File: | Description: |
---|---|
lib/TWiki/Contrib/DBCache.pm | Web cache class |
lib/TWiki/Contrib/Search.pm | Search object and parser |
lib/TWiki/Contrib/Search.yp | Source file for the parser. Search.pm can be generated by yapp -s Search.yp , if yapp is installed. |
lib/TWiki/Contrib/FileTime.pm | Cache support for remembering file dates |
lib/TWiki/Contrib/Array.pm | Array class |
lib/TWiki/Contrib/Map.pm | Map class |
lib/TWiki/Contrib/Archive.pm | Archiver used if Storable is not available |
lib/TWiki/Contrib/DBCacheContrib/build.pl | Build file |
lib/TWiki/Contrib/DBCacheContrib/test.zip | Test::Unit tests |
data/TWiki/DBCacheContrib.txt | Documentation topic |
DBCacheContrib_installer.pl | Install script |
DBCacheContrib.zip
in your twiki installation directory.
DBCacheContrib_installer
to automatically check and install other TWiki modules that this module depends on. You can also do this step manually.
Author: | TWiki:Main/CrawfordCurrie, TWiki:Main/ThomasWeigert | ||||||||||||
Copyright ©: | This code is based on an original development of Motorola Inc. and is protected by the following copyrights: Copyright © 2002-2003, 2005 Motorola Inc. All Rights Reserved. Portions copyright © 2004. Crawford Currie http://www.c-dot.co.uk | ||||||||||||
License: | As required for the publication of all extensions to TWiki, this software is published under the terms of the GNU General Public License. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details, published at http://www.gnu.org/copyleft/gpl.html | ||||||||||||
Version: | 1.002 | ||||||||||||
Change History: | 9 Jul 2005: Fixed updating of parents when reloading from m cache. 4 Jul 2005: Replaced date operators by the ability to leverage the SpreadSheetPlugin. 10 Jun 2005: Replaced parser with LALR-1 table driven parser. Store web information in cache. Fix summing of subfields, expansion of fields, [? search ] syntax, array notation. Empty search matches the whole data base (consistent with %SEARCH% ). | ||||||||||||
8 Jul 2004: Initial version, split out from FormQueryPlugin | |||||||||||||
Dependencies: |
| ||||||||||||
Perl Version: | 5.0 | ||||||||||||
Plugin Home: | http://TWiki.org/cgi-bin/view/Plugins/DBCacheContrib | ||||||||||||
Feedback: | http://TWiki.org/cgi-bin/view/Plugins/DBCacheContribDev | ||||||||||||
Demo URL: |
-- TWiki:Main/ThomasWeigert - 09 Jul 2005