Sunday, March 11, 2012

Replicate Salesforce Data to local MySql DB (Part 1)

Sorry about the long delay. During this time, i have switched between multiple technologies and recently involved heavily on the cloud based system like Google App Engine & Salesforce. While working on Salesforce i was looking for an application to download all my saleforce objects along with the data. There were lot of commercial app that does more than what i need with a bigger price tag. I also found some PHP based solutions fulfilling half of what i wanted. So i ended up writing my own application using Java with the following features, 

  • Should be an command line application so that i can schedule it. 
  • Should go through corporate firewall 
  • I should be able to stop and start 
  • I need to exclude objects that i don't want 
  • Multiple download of the same objects should be avoided during retries until a configured interval is passed.
I created my Java command line application with the following command line arguments,

usage: java -cp sfdata_duplicator.jar [-cdbs ] [-csql ] [-dbh ]
       [-dbp ] [-dbs ] [-dbu ] [-dte ] [-p ] [-pa
       ] [-pdb ] [-ph ] [-po ] [-pp ] [-pu ] [-sfe
       ] [-sfp ] [-sft ] [-sfu ] [-sp ] [-spr ]
       [-srel ] [-stbl ]
     -cdbs,--create-db-schema        Enable creating local schema from
                                           local sql files
     -csql,--create-local-sqlfiles   Enable creating local sql files from
     -dbh,--db-host                  Enter the mysql host name
     -dbp,--db-password              Enter the mysql password
     -dbs,--db-schema                Enter the mysql schema name
     -dbu,--db-user                  Enter the mysql user name
     -dte,--drop-table-if-exists     Drop table if already exists
     -p,--enable-proxy               Enable proxy[true|false]
     -pa,--proxy-auth-req            Is Proxy authentication required
     -pdb,--populate-db              Enable populating data from
     -ph,--proxy-host                Enter the proxy host name
     -po,--proxy-port                Enter the proxy host name
     -pp,--proxy-password            Enter the proxy password
     -pu,--proxy-user                Enter the proxy user name
     -sfe,--sf-endpoint              Enter the salesforce
                                           endpoint(default: sandbox)
     -sfp,--sf-password              Enter the salesforce password
     -sft,--sf-token                 Enter the salesforce token
     -sfu,--sf-user                  Enter the salesforce user name
     -sp,--sql-path                  Enter the path to store mysql queries
     -spr,--sql-prefix               Enter the prefix for creating tables
                                           in mysql
     -srel,--sql-relation-file       Enter the file name for storing
                                           relation queries
     -stbl,--sql-table-file          Enter the file name for storing table
End of GNU Help

I will go over other details in my next Part.


  1. This code looks useful. I'm a developer, but no experience with java or maven. Could you perhaps publish brief directions on how to run it? When I try to use maven to build it, I always get a java.lang.reflect.InvocationTargetException.

  2. This comment has been removed by the author.

  3. This might be a very basic question but i have tried configuring your application in eclipse and am having issue with configuring the arguments.
    could you provide a complete string of the command and the arguments like
    java -cp sfdata_duplicator.jar -cdbs -csql

  4. Can you provide a license on this? I'd like to post source to GitHub, with a few modifications. I need to know what license you'd apply to this before I consider posting my modified source.

    Thank you.

  5. Just put a link to this original content. You can also publish your link here so people can refer to your post when they come here.

  6. Thank you for your response. Would you agree that this lic fits?