Next, this plugin should come standard issue with WordPress. I know the keys are a bit of a hassle. But WordPress should offer something out or the box, because the amount of spam I get through the comment forms on this site is demoralizing.

Anyway, easy to install and well worth the effort, I hope. Find reCAPTCHA installation notes here:

http://wordpress.org/extend/plugins/wp-recaptcha/installation/

Now, let’s see if this thing works.

    function query_bottom_count($bottom_value, $start_uts, $end_uts)
    {
        $sql = <<<XSQL
SELECT
    REPLACE( SUBSTRING_INDEX( SUBSTRING_INDEX( last_result_url,
        '://', -1 ) , '/', 1 ) , 'www.', '' ) AS domain,
    COUNT( REPLACE( SUBSTRING_INDEX( SUBSTRING_INDEX( last_result_url, '://', -1 ) ,
        '/', 1 ) , 'www.', '' ) ) AS hits
FROM queries as Query
WHERE insert_uts >= ?
AND insert_uts < ?
GROUP BY domain HAVING hits = ?;
XSQL;

        $ParamList = array($start_uts, $end_uts, $bottom_value);
        $Data = $this->query($sql, $ParamList);
        if ( !$Data ) return 0;
        return count($Data);
    }

Just setting the record straight.

Cake Core Bugs

There are a couple documented bugs that still outstanding as of release 1.2.4.8284. If you are doing anything more than basic testing, one of these could bite you eventually:

https://trac.cakephp.org/ticket/6205
https://trac.cakephp.org/ticket/6468

Here is a patch file that fixes them:

# Fix issues with CakePhp model unit tests
# Tom at klenwell@gmail.com
# CakePhp Version: 1.2.4.8284

diff -r ead5d3b62da7 -r df04a99fec1b cake/libs/model/datasources/dbo_source.php
--- a/cake/libs/model/datasources/dbo_source.php	Mon Aug 10 21:16:09 2009 -0700
+++ b/cake/libs/model/datasources/dbo_source.php	Mon Sep 14 21:08:12 2009 -0700
@@ -2355,6 +2355,15 @@ class DboSource extends DataSource {
 			$column['default'] = null;
 		}

+                // solves fixture problem
+                // see https://trac.cakephp.org/ticket/6205
+                if (($column['type'] == 'datetime' || $column['type'] == 'timestamp' ) && isset($column['default']) && $column['default'] === '') {
+                    $column['default'] = null;
+                }
+                if ( $column['type'] == 'timestamp' && $column['default'] === 'CURRENT_TIMESTAMP' )
+                    #pr($column);
+                    $column['default'] = null;
+
 		if (isset($column['key']) && $column['key'] == 'primary' && $type == 'integer') {
 			$out .= ' ' . $this->columns['primary_key']['name'];
 		} elseif (isset($column['key']) && $column['key'] == 'primary') {
@@ -2455,4 +2464,4 @@ class DboSource extends DataSource {
 		return 'string';
 	}
 }
-?>
\ No newline at end of file
+?>

diff -r ead5d3b62da7 -r df04a99fec1b cake/tests/lib/cake_test_fixture.php
--- a/cake/tests/lib/cake_test_fixture.php	Mon Aug 10 21:16:09 2009 -0700
+++ b/cake/tests/lib/cake_test_fixture.php	Mon Sep 14 21:08:12 2009 -0700
@@ -114,7 +114,12 @@ class CakeTestFixture extends Object {
 			}
 		}

-		if (!isset($this->table)) {
+                // solves HABTM problem
+                // see https://trac.cakephp.org/ticket/6468
+                if (isset($model->table)) {
+                        $this->table = $model->table;
+                }
+                elseif (!isset($this->table)) {
 			$this->table = Inflector::underscore(Inflector::pluralize($this->name));
 		}

@@ -190,4 +195,4 @@ class CakeTestFixture extends Object {
 		return $return;
 	}
 }
-?>
\ No newline at end of file
+?>

Fixtures

The design and usage of fixtures still seems fickle. For instance, the CakePhp docs and packaged test examples promote the explicit declaration of fields in fixtures. I find this tedious. Happily, it is also unnecessary, thanks to the import property.

To avoid this nuisance, use the import ‘model’ or ‘table’ setting to automatically load the table schema from the existing database. See the example here:

simple_record_fixture.php

A couple other tips for using fixtures:

1. Import all fixtures associated with a model in the unit test for that model
2. If unit testing a plugin, the value in the fixtures property takes this format: plugin.$plugin_name.$model_name
3. Don’t forget to set up an empty database and include a test_suite setting in your database configuration file.

See these links for additional insight:
Cakewell Database Config File
Model Test with Fixtures
CakePhp cake_test_fixture.php

HABTM Relationships

To test models that have HABTM relationships with other models, it is necessary to create fixtures for both models having the HABTM relationship and a fixture to build the join table. See these files from the Cakewell project for examples:

Authwell User-Role HABTM Fixture
Authwell Role-Privilege HABTM Fixture

The Cakewell Authwell plugin include a good example of a complex model unit test that is successfully configured. It includes tests within the plugin directory that autoload a clean test database and successfully test models with HABTM associations. The source can be found here:

Authwell Plugin Test Directory

This post is mainly just to insure that everything is working correctly. More sporadic posting to follow.

The code is open source and available at the Cakewell Google Code site at:

http://code.google.com/p/cakewell/

The live demo can be found at:

http://cakewell.klenwell.com/

I’ll add updates here as I progress.

For more information on usage, I recommend the unit and acceptance tests that come packaged with the source code.

There are probably better PHP libraries for Google’s Data APIs (e.g. the Zend Framework.) PHPMailer, upon which my Gmail library was built, now supports Gmail natively. Google Analytics now has an export API. And PHP 4, of course, has reached the end of its life.

But Greqo 4 runs on PHP 5. And if you need a PHP 4 implementation of these Google APIs, well then here you are:

Download: Greqo 4 Release for Google Mail, Blogger, and Analytics

Going forward, the Greqo library will abandon complete PHP 4 compatibility and focus on PHP 5 and, eventually, PHP 6.

This is the official FormEncode explanation of state:

All the validators receive a magic, somewhat meaningless state argument (which defaults to None). It’s used for very little in the validation system as distributed, but is primarily intended to be an object you can use to hook your validator into the context of the larger system.

For instance, imagine a validator that checks that a user is permitted access to some resource. How will the validator know which user is logged in? State! Imagine you are localizing it, how will the validator know the locale? State! Whatever else you need to pass in, just put it in the state object as an attribute, then look for that attribute in your validator.

Hmmm. Maybe an example would help. Here’s the best example of its usage I could find on the Pylons site:

  this_schema = TheAppropriateSchema()
  try:
    state = dict() # this won't actually work. you'll need an object that formencode can hang things on.
    state["useful_state_information"] = something_useful
    form_result = this_schema.to_python(postvars, state)

    # validation is successful - form_result contains good data for you to consume

    redirect_to("somewhere")

  except Invalid, e:
    defaults = request.params
    errors = e.error_dict
    unfilled_html = a_function_that_draws_your_form_page_html()
    filled_html = htmlfill.render(unfilled_html, defaults, errors)
    return filled_html

Notice the comment: # this won’t actually work. Thanks for the warning.

So what is state and why would you want to use it?

Let’s answer the second question first, because at work, where we’re using the Pylons framework, I came up with an excellent situation which helped me figure out what it is and why I would want to use it.

The situation, briefly: our application has two separate forms. One is a form where a user can add new records in a multi-row table form. Each row is an individual record. The second is a review form, which looks just like the new record form, but is filled in from data uploaded in csv file (from another form).

Since DRY is a guiding principle of development in my office, the goal is to use the same underlying code for the form, controller, validation, etc. But the problem is that data originate in two different formats: one is as POST data from the add form, the second as the contents of a csv file. So how can we normalize these two data formats so that we can use the same underlying code? All together now: State! Now you’re talking!

If we could just tell our FormEncode validator where the data was coming from, then we could create two separate normalization methods in our validation class that would transform the data into a form that the validator and template could deal with. Anyway, enough verbiage. Find below a representative generalized version of the FormEncode validator I created and the controller it’s used in.

One more note before I finish. Notice the FormencodeState object comment. That explains why the example from the Pylons wiki does not work (in most cases). My first impulse would be to use a dict, too, as the state object. But the internals of the FormEncode validator class require an object. This is what gets passed in the validator method state argument and it is what passes info from the general framework environment to the validator.

Pylons Controller Code (with State Object)

class FormencodeState(object):
    """
    State class for formencode
    Although NOT well documented, to use the state argument in the to_python
    method in the context of schema that does complex, multistep validation,
    the state argument must be an object that formencode can hang additional
    attributes from, else you get errors like:
    Module formencode.schema:114 in _to_python
    >>  state.full_dict = value_dict
    <type 'exceptions.AttributeError'>: 'dict' object has no attribute 'full_dict'
    """
    pass

# ... within actual Controller class method
Validator = SourceDataPreValidator()
ControllerState = FormencodeState()
ControllerState.source = 'csv'
Validator.to_python(DataValue, ControllerState)     # <- state object in action!

FormEncode Validator Code

class SourceDataPreValidator(formencode.validators.FormValidator):
    """normalize data from either a form submission or a csv file upload and validate"""
    validate_partial_form = True
    def _to_python(self, value, state):
        """normalize csv input"""
        if state.source == 'csv':
            value['normal_data'] = self._normalize_csv_data(value.get('csv_import_file'), state)
        elif state.source == 'form':
            pass
        return value

    def _normalize_import_data(self, csv_file_string, state):
        NormalDataList = []
        ImportedLines = csv_file_string.split('\n')
        for i in range(len(ImportedLines)):
            ColValues = ImportedLines[i].split(',')
            fpre = 'csv_import-' + str(i)
            NormalDataList.append((fpre + '.id', str(ColValues[1])))
            NormalDataList.append((fpre + '.amount', str(ColValues[2])))
            NormalDataList.append((fpre + '.date',
                self._importdate_to_python_date(ColValues[4])))
        return NormalDataList

    def _importdate_to_python_date(self, datestr):
        """some code to convert a date string to object"""
        pass

I hope that sheds a little light on this powerful mystery. Questions or comments welcome.

What if I need to compare really big numbers in PHP? Like comparing 2^66 > 3^53?

Overview

This will work:

print (int) (pow(2,66) > pow(3,53));

PHP will convert the integers to scientific notation. But this script illustrates the limitation of normal operational syntax (i.e.: pow(2,66) > pow(3,53) vs. bccomp(bcpow(2,66), bcpow(3,53),1) > 0):

$max = 1000;
foreach ( range(1,$max) as $x )
{
    if (
        ! ( pow(2,$x)+1 > pow(2,$x) ) ||
        ! ( pow(2,$x)-1 < pow(2,$x) ) ||
        ! ( pow(2,$x-1)*2 == pow(2,$x) )
    ) break;
}

$maxpow = $x-1;
$maxint = bcpow(2, $maxpow);
$Result['op'] = "max supported int: 2^$maxpow or $maxint\n";

foreach ( range(1,$max) as $x )
{
    if (
        ! ( bccomp(bcadd(bcpow(2,$x),1), bcpow(2,$x)) == 1 ) ||
        ! ( bccomp(bcsub(bcpow(2,$x),1), bcpow(2,$x)) == -1 ) ||
        ! ( bccomp(bcmul(bcpow(2,$x-1),2), bcpow(2,$x)) == 0 )
    ) break;
}

$maxpow = $x-1;
$maxint = bcpow(2, $maxpow);
$Result['bc'] = "max supported int >= 2^$maxpow or $maxint\n";

printf("<pre>%s</pre>", print_r($Result,1));

Solution

Use bccomp. The script above can be found on the klenwell code site:

http://code.google.com/p/klenwell/source/browse/trunk/pastebin/bc_demo.php

Results will vary depending on the processing capacity of the system. Here’s the results on my machine:

In 5 Fairly Easy Steps

1. Install the needed programs

2. Install Thawte certificates for Gmail

EDIT: verisign.com apparently no longer issues certs at the address below. So the ‘wget’ step will not work. adkein, in a comment below (click here), links to another page when you can apparently get the needed cert. I cannot vouch for this as my cert still works. But you might try downloading that and putting it your ~/etc/.certs directory if you’re having issues finding the cert.

This is necessary for Gmail. (Thanks to laurentbois.com.)

3. Configure msmtp
This will open up a new msmtp configuration file

Copy the following lines. Replace UPPERCASE text with your personal settings:

# config options: http://msmtp.sourceforge.net/doc/msmtp.html#A-user-configuration-file
defaults
logfile /tmp/msmtp.log

# gmail account
#account gmail
auth on
host smtp.gmail.com
port 587
user YOURNAME@gmail.com
password YOURPASSWORD
from YOURNAME@gmail.com
tls on
tls_trust_file /home/USER/etc/.certs/ThawtePremiumServerCA.crt

# set default account to use (not necessary with single account)
#account default : gmail

Change permission on this file or msmtp will complain:

4. Configure mailx

Copy the following lines and replace UPPERCASE text with your personal settings:

# set smtp for nail
# ref: http://ubuntuforums.org/showpost.php?p=4531994&postcount=6

# gmail account (default)
# $ mailx -s "subject line" -a /path/file recipient@email.com < /path/body.txt
set from="YOURNAME@gmail.com (YOURNAME)"
set sendmail="/usr/bin/msmtp"
set message-sendmail-extra-arguments="-a gmail"

5. Run a test
Send a test message to your gmail account:

Check your gmail account and you should have a message from yourself. You can also check your log:

I personally prefer nail because it more easily accommodates multiple accounts. But if you don’t have a deliberate need to use multiple accounts and are using a later version of Ubuntu, use this.

This guide covers using the nail program, which is my client of choice but is no longer in the main repositories. Find a guide for using mailx, which is part of the current Ubuntu repositories, here.

In 6 Fairly Easy Steps

1. Add the breezy repositories containing nail to your source file
Edit your sources list

Add the following lines at bottom. These will enable you to apt-get nail

# breezy repositories (added to install nail)
# see http://old-releases.ubuntu.com/releases/ for more info
deb http://old-releases.ubuntu.com/ubuntu/ breezy universe

Don’t forget to update

2. Install the needed programs

3. Install Thawte certificates for Gmail

EDIT: verisign.com apparently no longer issues certs at the address below. So the ‘wget’ step will not work. According to this comment, you can downloaded the needed cert from http://www.cs.utexas.edu/~suriya/UT-wireless/ThawtePremiumServerCA_b64.txt. I cannot vouch for this as my cert still works. But you might try downloading that and putting it your ~/etc/.certs directory if you’re having issues finding the cert.

This is necessary for Gmail. (Thanks to laurentbois.com.)

4. Configure msmtp
This will open up a new msmtp configuration file

Copy the following lines. Replace UPPERCASE text with your personal settings:

# config options: http://msmtp.sourceforge.net/doc/msmtp.html#A-user-configuration-file
defaults
logfile /tmp/msmtp.log

# isp account
account isp
auth login
host SMTP.YOURISP.COM
port 25
user YOURNAME@ISP.COM
from YOURNAME@ISP.COM
password YOURPASSWORD

# gmail account
account gmail
auth on
host smtp.gmail.com
port 587
user YOURNAME@gmail.com
password YOURPASSWORD
from YOURNAME@gmail.com
tls on
tls_trust_file /home/USER/etc/.certs/ThawtePremiumServerCA.crt

# set default account to use (from above)
account default : gmail

Change permission on this file or msmtp will complain:

5. Configure nail

Copy the following lines and replace UPPERCASE text with your personal settings:

# set smtp for nail
# ref: http://ubuntuforums.org/showpost.php?p=4531994&postcount=6

# gmail account (default)
# $ nail -s "subject line" -a /path/file recipient@email.com < /path/body.txt
set from="YOURNAME@gmail.com (YOURNAME)"
set sendmail="/usr/bin/msmtp"
set message-sendmail-extra-arguments="-a gmail"

# isp account (add -A option to command line)
# $ nail -A isp -s "subject line" -a /path/file recipient@email.com < /path/body.txt
account isp {
set from="YOURNAME@ISP.COM"
set sendmail="/usr/bin/msmtp"
set message-sendmail-extra-arguments="-a isp"
}

6. Run a test
Send a test message to your gmail account:

Check your gmail account and you should have a message from yourself. You can also check your log:

That’s it. You should now be ready to incorporate email into your bash scripts, which is great for making backups. I did successfully test this on an old laptop had installed Ubuntu 8.04 this weekend.

If you have any problems or questions, I’d recommend posting to the ubuntu forums thread as you’ll probably get a quicker response there.