Differences between revisions 5 and 19 (spanning 14 versions)
Revision 5 as of 2009-09-09 01:49:48
Size: 694
Editor: c-67-176-12-8
Comment:
Revision 19 as of 2009-10-07 02:20:55
Size: 3036
Editor: c-67-176-12-8
Comment:
Deletions are marked like this. Additions are marked like this.
Line 4: Line 4:
 * Status (Tues 09-0802009)
Line 6: Line 5:
  * Currently I'm testing the use of the warm standby server as a PITR backup/recovery source (est completion = today) = Implementation Status =
== Updated Wed 09-09-2009 ==
Line 8: Line 8:
  * Next step is to finish the WAL backup scripts in a way that allows us to recover in a controlled manner - plus the creation of the backup meta data (est completion = Late tomorrrow) == Backup / Recovery ==
Line 10: Line 10:
  * Next I'll coordinate a full xfs backup/restore test on hmidb0/1 with xfsdump as the defined backup command (est completion = ? probably Thurs) 1. pitrBackup = 100% complete (We may still find bugs as we engage in further testing)
Line 12: Line 12:
  * After this I'll start on the "Kevin evaluates our plan to disseminate database data to the public..." task 2. walBackup = 100% complete (We may still find bugs as we engage in further testing)

3. Implement changes based on feedback if needed

4. Unit testing - 100% (Basic Unit tests)

5. Install 'Package” as a directory structure that will contain all future 'tools'
/bin, /etc, /tmp, /log, ...
Installed on hmidb (after the move to hmidb0 we'll setup the final packages)

6. Setup backup jobs (pending move to hmidb0)
Cron entry to run the base file system backup (I suggest once/quarter)
Cron entry to archive the WAL segments (monthly)
Optional additional base file system backups to alternate locations

The toolset will be run from the warm standby server


== Warm Standby ==

 * Documented strategy
  * Strategy fully dependant on the Backup / Recovery strategy and defined manual steps

  * We'll need to address the re-setup of SLONY after a fAIL over as part of the upcoming 'Data Replication' strategy

  * I successfully tested the backup of a running warm standby server and then recovering from the warm standby base backup and the subsequent archived WAL files from the master

  * walBackup script 100% complete
  
  * walBackup Unit Testing - 100% complete (Basic unit tests)

  * Still To Do: Revise / make more clear & easy to follow the recovery plan



== Web db Plan ==

 * Strategy documented - initial pass (100%)

== SLONY Plan ==
  
  * SLONY v2 must be used if we want to use PostgreSQL 8.4
  
  * SLONY-1 2.03 (release Candidate) recommended due to a few key bugs in 2.01 and 2.02

  * Next Steps:

    * Define SLONY Architecture

    * Define Log Shipping process

    * Build SLONY scripts/tools
 
    * Implement & Test



== Currently In Progress ==

   * Setup of a new (local) warm standby server (done)

   * further testing of the current backup scripts (done)

   * Design of the SLONY control modules (done)

   * Revise / make more clear & easy to follow the warm standby recovery plan (pending)

   * End2end testing:
      * setup 4 VM's (done)
      * install postgres on nodes 1 & 2 (done)
      * setup warm standby (pending)
      * install PITR backup scripts (done)
      * install slony control scripts (done)
      * setup initial slony cluster (pending, done & restarted several times already - testing mode)
      * test warm standby failover (pending)
      * test slony switchover / switch back (pending)
      * test pitr recovery (pending)
      * setup slony log shipping (pending)
      * test slony log shipping receiver (pending)
      * document end2end test results (pending)
      * add slony scripts to add/remove things from the slony cluster (pending)
      * test the add/remove things to slony script(s) (pending)

Kevins's archive

Anchor(mostrecent)

Implementation Status

Updated Wed 09-09-2009

Backup / Recovery

1. pitrBackup = 100% complete (We may still find bugs as we engage in further testing)

2. walBackup = 100% complete (We may still find bugs as we engage in further testing)

3. Implement changes based on feedback if needed

4. Unit testing - 100% (Basic Unit tests)

5. Install 'Package” as a directory structure that will contain all future 'tools' /bin, /etc, /tmp, /log, ... Installed on hmidb (after the move to hmidb0 we'll setup the final packages)

6. Setup backup jobs (pending move to hmidb0) Cron entry to run the base file system backup (I suggest once/quarter) Cron entry to archive the WAL segments (monthly) Optional additional base file system backups to alternate locations

The toolset will be run from the warm standby server

Warm Standby

  • Documented strategy
    • Strategy fully dependant on the Backup / Recovery strategy and defined manual steps
    • We'll need to address the re-setup of SLONY after a fAIL over as part of the upcoming 'Data Replication' strategy
    • I successfully tested the backup of a running warm standby server and then recovering from the warm standby base backup and the subsequent archived WAL files from the master
    • walBackup script 100% complete
    • walBackup Unit Testing - 100% complete (Basic unit tests)
    • Still To Do: Revise / make more clear & easy to follow the recovery plan

Web db Plan

  • Strategy documented - initial pass (100%)

SLONY Plan

  • SLONY v2 must be used if we want to use PostgreSQL 8.4
  • SLONY-1 2.03 (release Candidate) recommended due to a few key bugs in 2.01 and 2.02
  • Next Steps:
    • Define SLONY Architecture
    • Define Log Shipping process
    • Build SLONY scripts/tools
    • Implement & Test

Currently In Progress

  • Setup of a new (local) warm standby server (done)
  • further testing of the current backup scripts (done)
  • Design of the SLONY control modules (done)
  • Revise / make more clear & easy to follow the warm standby recovery plan (pending)

  • End2end testing:
    • setup 4 VM's (done)
    • install postgres on nodes 1 & 2 (done)

    • setup warm standby (pending)
    • install PITR backup scripts (done)
    • install slony control scripts (done)
    • setup initial slony cluster (pending, done & restarted several times already - testing mode)

    • test warm standby failover (pending)
    • test slony switchover / switch back (pending)
    • test pitr recovery (pending)
    • setup slony log shipping (pending)
    • test slony log shipping receiver (pending)
    • document end2end test results (pending)
    • add slony scripts to add/remove things from the slony cluster (pending)
    • test the add/remove things to slony script(s) (pending)

JsocWiki: KevinsProgress (last edited 2013-05-01 04:35:27 by localhost)