Sunday, March 11, 2012

Replicate Salesforce Data to local MySql DB (Part 1)

Sorry about the long delay. During this time, i have switched between multiple technologies and recently involved heavily on the cloud based system like Google App Engine & Salesforce. While working on Salesforce i was looking for an application to download all my saleforce objects along with the data. There were lot of commercial app that does more than what i need with a bigger price tag. I also found some PHP based solutions fulfilling half of what i wanted. So i ended up writing my own application using Java with the following features, 

  • Should be an command line application so that i can schedule it. 
  • Should go through corporate firewall 
  • I should be able to stop and start 
  • I need to exclude objects that i don't want 
  • Multiple download of the same objects should be avoided during retries until a configured interval is passed.
I created my Java command line application with the following command line arguments,

usage: java -cp sfdata_duplicator.jar [-cdbs ] [-csql ] [-dbh ]
       [-dbp ] [-dbs ] [-dbu ] [-dte ] [-p ] [-pa
       ] [-pdb ] [-ph ] [-po ] [-pp ] [-pu ] [-sfe
       ] [-sfp ] [-sft ] [-sfu ] [-sp ] [-spr ]
       [-srel ] [-stbl ]
     -cdbs,--create-db-schema        Enable creating local schema from
                                           local sql files
     -csql,--create-local-sqlfiles   Enable creating local sql files from
     -dbh,--db-host                  Enter the mysql host name
     -dbp,--db-password              Enter the mysql password
     -dbs,--db-schema                Enter the mysql schema name
     -dbu,--db-user                  Enter the mysql user name
     -dte,--drop-table-if-exists     Drop table if already exists
     -p,--enable-proxy               Enable proxy[true|false]
     -pa,--proxy-auth-req            Is Proxy authentication required
     -pdb,--populate-db              Enable populating data from
     -ph,--proxy-host                Enter the proxy host name
     -po,--proxy-port                Enter the proxy host name
     -pp,--proxy-password            Enter the proxy password
     -pu,--proxy-user                Enter the proxy user name
     -sfe,--sf-endpoint              Enter the salesforce
                                           endpoint(default: sandbox)
     -sfp,--sf-password              Enter the salesforce password
     -sft,--sf-token                 Enter the salesforce token
     -sfu,--sf-user                  Enter the salesforce user name
     -sp,--sql-path                  Enter the path to store mysql queries
     -spr,--sql-prefix               Enter the prefix for creating tables
                                           in mysql
     -srel,--sql-relation-file       Enter the file name for storing
                                           relation queries
     -stbl,--sql-table-file          Enter the file name for storing table
End of GNU Help

I will go over other details in my next Part.