ipfm2psql
--------

This is a Perl script that 
 - reads a directory containing IPFM log files,
 - parses each file,
 - stores the data from the file into a PostgreSQL database,
 - deletes the file

Just dump the necessary configuration into /etc/ipfm2psql.conf (based on
the ipfm2psql.conf.sample file in this directory), set up the database,
do a test run and stuff it into a suitable crontab.


Migrating from older ipfm2psql versions to 0.5
----------------------------------------------

With ipfm2psql 0.5 the database structure has changed. If you wish to keep
your old trafficlog data, you have to perform a data migration. There are two
ways to do this:
 - online migration:
 - offline migration using dumps

Online migration only half tested and not recommended. The enclosed migrate.pl
script should do most of the work.

The recommended way is offline migration.

Offline migration: 
 - stop and disabled the jobs loading new data into the database,
 - dump the existing data (add host/user parameters as needed):
    pg_dump --data-only ipfm > ipfm.sql
 - rewrite the dumped data into the new format
   cat ipfm.sql | ./dumpmigrate.pl
 - you will end up with two files:
   - hosts-dump.sql,
   - traffic-dump.sql
 - drop the existing database (you _did_ keep the data only dumps, did you?),
 - recreate the database,
 - load the new database schema into it from ipfm.sql,
 - load the converted dumps using loaddump.pl, adjust the content of
   the %to hash in this file first - replace the placeholders with the
   correct data for your environment
   - ./loaddump.pl
 - update the sequences:
   - connect to the ipfm database using psql,
   - hosts:
     - execute: SELECT max(id) from hosts 
     - the number you get is the maximum id for the hosts (OLDMAX), keep it
     - compute NEWMAX = OLDMAX + 1
     - execute the following statement, replacing NEWMAX with its previously
       computed value:
         SELECT pg_catalog.setval ('hosts_id_seq', NEWMAX, true)
 - Congratulations, the data migration is complete!
 - update the ipfm2psql script your are using to the current version,
 - restart and re-enable the jobs loading new data into the database
 - DONE. 
