I'm using python to connect to a MySQL database, loop through a list of table names and then create csv files from those tables. (End game is to geocode the csv tables in another process...)
I need to pass a standard sql select expression, but I'm having a devil of a time getting it the work correctly. My code:
<SPAN class="keyword token">def</SPAN> <SPAN class="token function">createCSV</SPAN><SPAN class="punctuation token">(</SPAN>table<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
cn <SPAN class="operator token">=</SPAN> mysql<SPAN class="punctuation token">.</SPAN>connect<SPAN class="punctuation token">(</SPAN>user <SPAN class="operator token">=</SPAN> <SPAN class="string token">'myUser'</SPAN><SPAN class="punctuation token">,</SPAN> password <SPAN class="operator token">=</SPAN> <SPAN class="string token">'myPassword'</SPAN><SPAN class="punctuation token">,</SPAN> host <SPAN class="operator token">=</SPAN> <SPAN class="string token">'myHost'</SPAN><SPAN class="punctuation token">,</SPAN> database <SPAN class="operator token">=</SPAN> <SPAN class="string token">'master_db'</SPAN><SPAN class="punctuation token">)</SPAN>
cursor <SPAN class="operator token">=</SPAN> cn<SPAN class="punctuation token">.</SPAN>cursor<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN>
<SPAN class="comment token">#query = f'"SELECT * FROM master_db.{table}"'</SPAN>
query <SPAN class="operator token">=</SPAN> <SPAN class="string token">'"SELECT * FROM master_db.{}"'</SPAN><SPAN class="punctuation token">.</SPAN>format<SPAN class="punctuation token">(</SPAN>table<SPAN class="punctuation token">)</SPAN>
cursor<SPAN class="punctuation token">.</SPAN>execute<SPAN class="punctuation token">(</SPAN>query<SPAN class="punctuation token">)</SPAN>
rows <SPAN class="operator token">=</SPAN> cursor<SPAN class="punctuation token">.</SPAN>fetchall<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN>
column_names <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN>i<SPAN class="punctuation token">[</SPAN><SPAN class="number token">0</SPAN><SPAN class="punctuation token">]</SPAN> <SPAN class="keyword token">for</SPAN> i <SPAN class="keyword token">in</SPAN> cursor<SPAN class="punctuation token">.</SPAN>description<SPAN class="punctuation token">]</SPAN>
dotcsv <SPAN class="operator token">=</SPAN> f<SPAN class="string token">'{table}.csv'</SPAN>
outFile <SPAN class="operator token">=</SPAN> open<SPAN class="punctuation token">(</SPAN>os<SPAN class="punctuation token">.</SPAN>path<SPAN class="punctuation token">.</SPAN>join<SPAN class="punctuation token">(</SPAN>outFolder<SPAN class="punctuation token">,</SPAN>dotcsv<SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'w'</SPAN><SPAN class="punctuation token">,</SPAN>newline <SPAN class="operator token">=</SPAN><SPAN class="string token">''</SPAN><SPAN class="punctuation token">)</SPAN>
myFile <SPAN class="operator token">=</SPAN> csv<SPAN class="punctuation token">.</SPAN>writer<SPAN class="punctuation token">(</SPAN>outFile<SPAN class="punctuation token">)</SPAN>
myFile<SPAN class="punctuation token">.</SPAN>writerow<SPAN class="punctuation token">(</SPAN>column_names<SPAN class="punctuation token">)</SPAN>
myFile<SPAN class="punctuation token">.</SPAN>writerows<SPAN class="punctuation token">(</SPAN>rows<SPAN class="punctuation token">)</SPAN>
cursor<SPAN class="punctuation token">.</SPAN>close<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN>
<SPAN class="keyword token">def</SPAN> <SPAN class="token function">main</SPAN><SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN><SPAN class="punctuation token">:</SPAN>
tableList <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN><SPAN class="string token">'businesslicenses'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'businesslicenseenforcements'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'complaints'</SPAN><SPAN class="punctuation token">,</SPAN>
<SPAN class="string token">'buildingpermits'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'inspections'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'codeenforcementcases'</SPAN><SPAN class="punctuation token">]</SPAN>
<SPAN class="keyword token">for</SPAN> table <SPAN class="keyword token">in</SPAN> tableList<SPAN class="punctuation token">:</SPAN>
table <SPAN class="operator token">=</SPAN> table<SPAN class="punctuation token">.</SPAN>replace<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"'"</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">""</SPAN><SPAN class="punctuation token">)</SPAN>
createCSV<SPAN class="punctuation token">(</SPAN>table<SPAN class="punctuation token">)</SPAN>
<SPAN class="keyword token">if</SPAN> __name__ <SPAN class="operator token">==</SPAN> <SPAN class="string token">'__main__'</SPAN><SPAN class="punctuation token">:</SPAN> <SPAN class="comment token">#if def main exists, call it</SPAN>
main<SPAN class="punctuation token">(</SPAN><SPAN class="punctuation token">)</SPAN> <SPAN class="line-numbers-rows"><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN></SPAN>The problem is coming from lines 7 or 8; I've tried both f string and .format(). I get the following error:
ProgrammingError<SPAN class="punctuation token">:</SPAN> You have an error <SPAN class="keyword token">in</SPAN> your SQL syntax<SPAN class="punctuation token">;</SPAN>
check the manual that corresponds to your MySQL server
version <SPAN class="keyword token">for</SPAN> the right syntax to use near
<SPAN class="string token">'"SELECT * FROM master_db.businesslicenses"'</SPAN> at line <SPAN class="number token">1</SPAN><SPAN class="line-numbers-rows"><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN></SPAN>
When I hard code the the select expression to "SELECT * FROM master_db.businesslicenses" rather than using the for loop, it runs just fine.
If I modify the for loop and put a print() function instead of the call to my createCSV() def, no errors:
tableList <SPAN class="operator token">=</SPAN> <SPAN class="punctuation token">[</SPAN><SPAN class="string token">'businesslicenses'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'businesslicenseenforcements'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'complaints'</SPAN><SPAN class="punctuation token">,</SPAN>
<SPAN class="string token">'buildingpermits'</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">'inspections'</SPAN><SPAN class="punctuation token">,</SPAN> <SPAN class="string token">'codeenforcementcases'</SPAN><SPAN class="punctuation token">]</SPAN>
<SPAN class="keyword token">for</SPAN> table <SPAN class="keyword token">in</SPAN> tableList<SPAN class="punctuation token">:</SPAN>
table <SPAN class="operator token">=</SPAN> table<SPAN class="punctuation token">.</SPAN>replace<SPAN class="punctuation token">(</SPAN><SPAN class="string token">"'"</SPAN><SPAN class="punctuation token">,</SPAN><SPAN class="string token">""</SPAN><SPAN class="punctuation token">)</SPAN>
<SPAN class="comment token">#createCSV(table)</SPAN>
query <SPAN class="operator token">=</SPAN> f<SPAN class="string token">'"SELECT * FROM master_db.{table}"'</SPAN>
<SPAN class="keyword token">print</SPAN><SPAN class="punctuation token">(</SPAN>query<SPAN class="punctuation token">)</SPAN>
<SPAN class="string token">"SELECT * FROM master_db.businesslicenses"</SPAN>
<SPAN class="string token">"SELECT * FROM master_db.businesslicenseenforcements"</SPAN>
<SPAN class="string token">"SELECT * FROM master_db.complaints"</SPAN>
<SPAN class="string token">"SELECT * FROM master_db.buildingpermits"</SPAN>
<SPAN class="string token">"SELECT * FROM master_db.inspections"</SPAN>
<SPAN class="string token">"SELECT * FROM master_db.codeenforcementcases"</SPAN><SPAN class="line-numbers-rows"><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN><SPAN></SPAN></SPAN>Obviously lines 9 through 14 are the output from my print(query) function and the select strings look good to me.
What am I missing?