Loader_Generate_Script
Prev | 14.2. Tiger Geocoder | Next

## Name

Loader_Generate_Script — Generates a shell script for the specified platform for the specified states that will download Tiger data, stage and load into `tiger_data` schema. Each state script is returned as a separate record. Latest version supports Tiger 2010 structural changes and also loads census tract, block groups, and blocks tables.

## Synopsis

`setof text loader_generate_script(`text[] param_states, text os`)`;

## Description

Generates a shell script for the specified platform for the specified states that will download Tiger data, stage and load into `tiger_data` schema. Each state script is returned as a separate record.

It uses unzip on Linux (7-zip on Windows by default) and wget to do the downloading. It uses Section 4.7.2, “Using the Shapefile Loader” to load in the data. Note the smallest unit it does is a whole state, but you can overwrite this by downloading the files yourself. It will only process the files in the staging and temp folders.

It uses the following control tables to control the process and different OS shell syntax variations.

1. `loader_variables` keeps track of various variables such as census site, year, data and staging schemas

2. `loader_platform` profiles of various platforms and where the various executables are located. Comes with windows and linux. More can be added.

3. `loader_lookuptables` each record defines a kind of table (state, county), whether to process records in it and how to load them in. Defines the steps to import data, stage data, add, removes columns, indexes, and constraints for each. Each table is prefixed with the state and inherits from a table in the tiger schema. e.g. creates `tiger_data.ma_faces` which inherits from `tiger.faces`

Availability: 2.0.0 to support Tiger 2010 structured data and load census tract (tract), block groups (bg), and blocks (tabblocks) tables .

If you are using pgAdmin 3, be warned that by default pgAdmin 3 truncates long text. To fix, change File -> Options -> Query Tool -> Query Editor - > Max. characters per column to larger than 50000 characters.

## Examples

Using psql where gistest is your database and `/gisdata/data_load.sh` is the file to create with the shell commands to run.

    psql -U postgres -h localhost -d gistest -A -t \
     -c "SELECT Loader_Generate_Script(ARRAY['MA'], 'gistest')" > /gisdata/data_load.sh;

Generate script to load up data for 2 states in Windows shell script format.

    SELECT loader_generate_script(ARRAY['MA','RI'], 'windows') AS result;
    -- result --
    set TMPDIR=\gisdata\temp\
    set UNZIPTOOL="C:\Program Files\7-Zip\7z.exe"
    set WGETTOOL="C:\wget\wget.exe"
    set PGBIN=C:\Program Files\PostgreSQL\9.4\bin\
    set PGPORT=5432
    set PGHOST=localhost
    set PGUSER=postgres
    set PGPASSWORD=yourpasswordhere
    set PGDATABASE=geocoder
    set PSQL="%PGBIN%psql"
    set SHP2PGSQL="%PGBIN%shp2pgsql"
    cd \gisdata

    cd \gisdata
    %WGETTOOL% ftp://ftp2.census.gov/geo/tiger/TIGER2015/PLACE/tl_*_25_* --no-parent --relative --recursive --level=2 --accept=zip --mirror --reject=html
    cd \gisdata/ftp2.census.gov/geo/tiger/TIGER2015/PLACE
    :
    :

Generate sh script

    SELECT loader_generate_script(ARRAY['MA','RI'], 'sh') AS result;
    -- result --
    TMPDIR="/gisdata/temp/"
    UNZIPTOOL=unzip
    WGETTOOL="/usr/bin/wget"
    export PGBIN=/usr/lib/postgresql/9.4/bin
    -- variables used by psql: https://www.postgresql.org/docs/current/static/libpq-envars.html
    export PGPORT=5432
    export PGHOST=localhost
    export PGUSER=postgres
    export PGPASSWORD=yourpasswordhere
    export PGDATABASE=geocoder
    PSQL=${PGBIN}/psql
    SHP2PGSQL=${PGBIN}/shp2pgsql
    cd /gisdata

    cd /gisdata
    wget ftp://ftp2.census.gov/geo/tiger/TIGER2015/PLACE/tl_*_25_* --no-parent --relative --recursive --level=2 --accept=zip --mirror --reject=html
    cd /gisdata/ftp2.census.gov/geo/tiger/TIGER2015/PLACE
    rm -f ${TMPDIR}/*.*
    :
    :

## See Also

Section 2.4.1, “Tiger Geocoder Enabling your PostGIS database: Using Extension”, Loader_Generate_Nation_Script, Drop_State_Tables_Generate_Script

Prev | Up | Next
Loader_Generate_Census_Script | Home | Loader_Generate_Nation_Script