Importing Building Data into ROVER

Prepared for
Federal Emergency Management Agency
Mitigation Directorate
Washington, DC
USA

By
Applied Technology Council
Redwood City, CA

Version 7
17 Sep 2013

Project ATC-67-5
FEMA 154 Rapid Visual Screening: Seismic Mobile Technology
 TABLE OF CONTENTS

1. Create the Data File ...................................................................................................... 2

2. Save the Data File and the Python Import Script in the Same Folder .......................... 3
3. Install ROVER or Know the URL and Credentials for a Host ..................................... 8
4. Import your Data, Case 1: Using ROVER Server on Your Own PC ........................... 8
5. Import your Data, Case 2: Using a Service Provider .................................................. 15

LIST OF FIGURES

Figure 1. Sample contents of the ROVERdata folder ......................................................... 8

Figure 2. The ROVER server icon as it appears on the desktop of a Windows 7 PC ........ 9
Figure 3. ROVER Server start dialog ................................................................................. 9
Figure 4. ROVER Server user login screen ...................................................................... 10
Figure 5. ROVER Server system selection ....................................................................... 10
Figure 6. ROVER Pre-earthquake server main menu ....................................................... 11
Figure 7. How to find the Windows command prompt in Windows 7 ............................. 12
Figure 8. The Windows command prompt in Windows 7 ................................................ 12
Figure 9. How to change to the ROVERdata directory .................................................... 13
Figure 10. Command prompt after you have successfully changed directory .................. 13
Figure 11. Entering the import command at the command prompt .................................. 14
Figure 12. The command prompt after the successful import of 1 building..................... 14
Figure 13. ROVER pre-earthquake server site list after successful import ...................... 15
Figure 14. How to exit the command prompt ................................................................... 15
Figure 15. Logging into a service provider's ROVER server ........................................... 16
Figure 16. ROVER Server system selection ..................................................................... 16
Figure 17. Getting the list of sites from the pre-earthquake server main menu................ 17
Figure 18. How to find the Windows command prompt in Windows 7 ........................... 18
Figure 19. The Windows command prompt in Windows 7 .............................................. 18
Figure 20. How to ping the ROVER server ...................................................................... 19
Figure 21. How to change to the ROVERdata directory .................................................. 19
Figure 22. Command prompt after you have successfully changed directory .................. 20
Figure 23. Entering the command to import the building data ......................................... 20
Figure 24. The command prompt after successfully importing the building data ............ 21
Figure 25. Pre-earthquake server site list after successfully importing building data ...... 21
Figure 26. How to exit the Windows command prompt ................................................... 22

LIST OF TABLES

Table 1. Building-data fields that one can import into ROVER ......................................... 3

1
 How to Import Basic Building Data into ROVER
These instructions guide the ROVER user through the process of importing basic building data
(addresses, possibly number of stories and other information) into ROVER. This is for the user
who has such data in an electronic format and wants to avoid retyping it all in by hand. We
explain here how to create a text file with the data in it, save the data file, start the Windows
command-line prompt, and type a command that will cause ROVER to import the data in the text
file.

It would be preferable to have a simple user interface or wizard in ROVER to walk the user
through this process and import the data file without going through the Windows command line.
We hope to add such a feature in the future. For now, the user who is moderately adept with
Window should be able to perform these tasks. They generally only need to be performed once,
so these steps should not be a burden to most users.

The general steps are to create the data file, save it and a Python script that is provided to you in
the same directory, install ROVER on your own PC or know the URL and credentials for a host,
and then import the data. The steps are detailed below.

1. Create the Data File

1. Start Excel or other spreadsheet software, and open a new file. The first row will have
column headers in it that explain what is in each row. Each row will have the data for one
building.
2. In row 1, type the column headers, one column header in each column starting from A1. The
available column headers are shown in Table 1, below. The table shows the available column
headers. It also , an example of an entry you put under that header for a building, a code that
we will explain and use later to tell ROVER which data fields you are using, and comments
that further explain what each field means. Note that you only need to type the column
headers for the columns you have data for. If you dont have latitude and longitude for
example, dont put those headers in the first row, just enter the headers for which you have
data. You dont have to have empty columns for data you dont have. For example, if you
only have address, city, state, ZIP Code, and number of stories, type Address (without the
quotes) in cell A1, City in cell B1, State in C1, ZIP Code in D1, and Number of
stories in E1.

Then in row 2, enter the data about the 1st building in your database. Put the street address in
the column labeled Address, the city name in the column labeled City, etc. Note that you can
enter street addresses in one of two ways (but not both): in a single field, such as 2373
Central Park Blvd under Address or as several fields, such as 2373 under Address
number, Central Park under Address street name, and Blvd under Address street type.
Dont do both, and dont switch back and forth, one row one way, the next row the other
way.

2
Table 1. Building-data fields that one can import into ROVER

Column Example Column Comment

header code
Address 2373 Central A You can enter the street address this way or by parts, as
Park Blvd shown under Address number, Address direction, etc.
City Denver C
State CO S
ZIP Code 80238 Z ZIP Code is required
Other IDs Smartspace O
Latitude 41.1869 L Enter as decimal degrees north, not degrees minutes
seconds, not degrees and decimal minutes, etc. Dont put in
the degrees symbol (as in 41.1869) and dont put in the
direction (as in 41.1869N)
Longitude -96.0476 G Enter as decimal degrees east of Greenwich, which means
addresses in the USA have negative longitude. Again, no
degrees symbol, no direction, no degrees-minutes-seconds.
Building SPA Risk LLC N
name
Number of 1 f Number of stories at ground level and above
stories
Year built 1915 Y Can be approximate.
Address 2373 n Enter the street address this way, by parts of the street
number address, or as a whole, as shown above in the line labeled
Address, but not both, and dont switch back and forth.
Address D This would be the direction NW for the address 8000 NW
direction Colorado Blvd
Address Central Park S Ditto
street name
Address Blvd T
street type
Address Z This would be the direction NW for the address 8000
street Colorado Blvd NW
direction

2. Save the Data File and the Python Import Script in the
Same Folder
When you are done typing or pasting data into the text file, do the following.

3. Delete any empty rows between two rows that do have data.
4. Delete empty columns between columns that do have data.
5. Delete any text that isnt a header in row 1 or data in rows 2 and onward under a header.
6. Save the file as a comma-separated value text file someplace where you can easily find it
again, such as in a folder named ROVER data on your desktop with the file name

3
 addresses.csv. In Excel 2010, the command would be File, Save as, and in the drop-
down box labeled Save as type, select CSV (Comma delimited). Give the file name
addresses and the directory as Desktop \ ROVERdata.
7. Box 1 shows the contents of a sample addresses.csv data file created with Microsoft Excel:

Box 1. Sample contents of addresses.csv

Address,City,State,ZIP Code,Other IDs,Latitude,Longitude,Building

name,Number of stories,Year built
2373 Central Park Blvd,Denver,CO,80238,Smartspace,41.1869,-
96.0476,SPA Risk LLC,1,2005

8. Write down the column codes for the fields you have in your data file. See Column code in
the table above. For example, if you have address, city, state, ZIP Code, other IDs, latitude,
longitude, building name, number of stories, and year built, write down ACSZOLGNfY
(without the quotes) on a piece of paper or a notepad file. In this example, A is for address, C
is for city, S is for state, Z is for ZIP Code, etc., as shown in the table.
9. Get the Python script file bulk_buildings.py. Place it in the same folder as your addresses file
addresses.csv. There is a printout of the Python script in Box 2; it can be copied and pasted in
a text file named bulk_buildings.py. Figure 1 shows sample contents of the folder with the
script and the addresses text file.
10. Note that these instructions are intended to make the import process as simple and reliable as
possible. ROVER is more flexible than is implied here. It is not absolutely necessary to
delete empty columns; they will be ignored. If you are not specifically creating the address
data file just for this purpose, for example you already have a spreadsheet with these data,
then in the with column codes mentioned in Step 8, you can use any unused letter such as an
underscore to designate a column to be ignored.

Box 2. Contents of Python script file "bulk_buildings.py"

from optparse import OptionParser

import csv
import urllib2
import urllib
import xml.parsers.expat
import string
import traceback
import xml.sax.saxutils

desc_text = "Reads a CSV file as might be generated by Excel and

add the building defined by each line to a ROVER Server."
parser = OptionParser( description = desc_text)
parser.add_option("-f", "--file",
help="(REQUIRED) read building info from csv
file FILE", metavar="FILE")
parser.add_option("-u", "--user", dest="screener",

4
 help="(REQUIRED) username for ROVER Server
admin account")
parser.add_option("-p", "--password",
help="(REQUIRED) password for ROVER Server
admin account")
parser.add_option("-s", "--server",
help="(REQUIRED) URL for ROVER Server")
parser.add_option("-c", "--columns",
help="(REQUIRED) column mapping, one letter
per column: [A]ddress, [C]ity, [S]tate, [Z]ipcode, [O]ther IDs,
[L]atitude, Lon[G]itude, [N]ame, \
Number o[f] stories, [Y]ear built, Address [n]umber, Address
[d]irection, Address [s]treet name, Address street [t]ype,
Address street [D]ir; any other letter means that column is
ignored")
parser.add_option("-x", "--skip_header",
action="store_true", dest="skip_header",
default=True,
help="skip first line (assumed to be
headers)")

(options, args) = parser.parse_args()

for k in ("file","screener","password","server","columns"):
if getattr(options,k) == None:
print "Argument '%s' is required" % (k if k!="screener"
else "user")
exit()
argval = {"zipcode":"", "other_ids":"","address":"","city":"",
"state":"", "name":"", "num_stories":"", "year_built":"",
"latitude":"", "longitude":"", "fnum":"", "fdir":"",
"sname":"", "stype":"", "sdir":""};
argname =
{"A":"address","C":"city","S":"state","N":"name",'Z':'zipcode','
O':'other_ids', "f":"num_stories", "Y":"year_built",

'L':'latitude','G':'longitude','n':'fnum','d':'fdir','s':'sname'
,'t':'stype','D':'sdir'}
for c in "CSN":
if options.columns.count(c) != 1:
print "columns argument must include exactly 1 '%s'" % c
exit()
ac = options.columns.count('A')
if ac == 0:
for c in "ndstD":
if options.columns.count(c) > 1:

5
 print "columns argument must include at most 1 '%s'"
% c
exit()
elif ac != 1:
print "columns argument must include at most 1 'A'"
exit()
else:
for c in "ndstD":
if options.columns.count(c) > 1:
print "columns argument has an 'A', so will ignore
'%s'" % c
for c in "OZLGfY":
if options.columns.count(c) > 1:
print "columns argument must include at most 1 '%s'" % c
exit()
if options.columns.count(c) == 0:
del argname[c]
fns = [(x if x in "ACSNOZLGndstDfY" else None) for x in
options.columns]
spamReader = csv.DictReader(open(options.file),
dialect="excel",fieldnames=fns)
first = True
rownum = 1
added = 0
for row in spamReader:
if options.skip_header and first:
first = False
continue
for k,v in argname.items():
if k not in row:
continue
argval[v] = xml.sax.saxutils.escape(row[k].strip()) #
urllib.quote( row[k].strip() )
#data = open(options.file).read()
#print data
url = "http://%s/Rover/client/siteupload?screener=%s&pw=%s"
% (options.server,urllib.quote(options.screener),
urllib.quote(options.password))
#print url
if ac == 0:
argval['address'] = ("%s %s %s %s %s" % (argval['fnum'],
argval['fdir'], argval['sname'], argval['stype'],
argval['sdir']) )
if argval['address'].strip() == argval['city'].strip():
argval['city'] = ''
for k in ('num_stories', 'year_built'):
try:

6
 argval[k] = "%d" % int(round(float(argval[k])))
except:
print 'row %d: bad %s: %s' % (rownum, k, argval[k])
argval[k] = ''
data = """<?xml version="1.0" encoding="utf-8"
standalone="yes"?>
<atc67site>
<version>0.0.1</version>
<screener>
<name>""" + options.screener + """</name>
</screener>
<building>
<buildingID></buildingID>
<name>""" + argval['name'] + """</name>
<address>""" + argval['address'] + """</address>
<city>""" + argval['city'] + """</city>
<state>""" + argval['state'] + """</state>
<zipcode>""" + argval['zipcode'] + """</zipcode>
<area></area>
<latitude>""" + argval['latitude'] + """</latitude>
<longitude>""" + argval['longitude'] + """</longitude>
<otheridentifiers>""" + argval['other_ids'] +
"""</otheridentifiers>
<numstories>""" + argval['num_stories'] + """</numstories>
<yearconstruction>""" + argval['year_built'] +
"""</yearconstruction>
<occupancyclasses></occupancyclasses>
<scores>
</scores>
</building>
</atc67site>"""
req = urllib2.Request(url, data)
try:
response = urllib2.urlopen(req)
msg = response.read()
added += 1
except Exception as e:
print "Building defined in row %d failed" % rownum
print e
print data
rownum += 1
if rownum % 100 == 0:
print "row %d processed" % rownum
print "%d (of %d possible) buildings added" % (added, rownum-1)

7
Figure 1. Sample contents of the ROVERdata folder

3. Install ROVER or Know the URL and Credentials for a Host

Before you import the data file, make sure you have done the following:

11. Install the ROVER server or contract with a service provider who will host the data for you.
A recent copy, possibly newer than the official FEMA release, can be found at
http://www.roverready.org.
12. Know the URL for your ROVER server. If you are using ROVER on your own PC, the URL
is probably http://localhost:8000. If a service provider is hosting your data for you, the
service provider should have given you a URL, such as http://testbed.isti.com/utah.
13. Know your admin username and password. For example, the default when you first install
ROVER are username = admin, password = rover.

4. Import your Data, Case 1: Using ROVER Server on Your

Own PC
14. If you are using the ROVER Server on your own PC, start the ROVER server. Commonly
there is an icon of the ROVER mascot (a dog) on your desktop labeled ROVERServer.
Click it. For example, see Figure 2:

8
Figure 2. The ROVER server icon as it appears on the desktop of a Windows 7 PC

You should now see something like Figure 3 (though the numbers in the dialog boxes may be
different):

Figure 3. ROVER Server start dialog

15. Write down the numeric IP address that appears in the dialog box to the right of Running
from host: and the port number that appears in the dialog box to the right of Running from
port: In the previous example, the host IP address is 192.168.0.13 and the port is 8000.
16. Press start server. Your browser should now open automatically, as shown in Figure 4
(though your browser may look different):

9
Figure 4. ROVER Server user login screen

17. Enter your username and password and press the login button. The default username and
password are admin and rover, respectively. If you changed them when you installed
ROVER, enter the new credentials.

18. Click Select FEMA 154 (pre-earthquake), as shown in Figure 5.

Figure 5. ROVER Server system selection

10
19. Click View list of active sites and their worksheets, as shown in Figure 6.

Figure 6. ROVER Pre-earthquake server main menu

20. Open the command prompt. In Windows 7 for example, touch the Start button, select All
programs, navigate to the Accessories folder, and click Command prompt, as shown in
Figure 7. That will open a little window where you can type the command prompt, as shown
in Figure 8.

11
Figure 7. How to find the Windows command prompt in Windows 7

Figure 8. The Windows command prompt in Windows 7

12
21. Change directories to the folder where you placed the script and data file your Desktop by
typing CD Desktop\ROVERdata at the command prompt and pressing Enter, as shown in
Figure 9.

Figure 9. How to change to the ROVERdata directory

You should now see the folder name in the command prompt, as shown in Figure 10.

Figure 10. Command prompt after you have successfully changed directory

22. Type the following command (Box 3) after the command prompt, as shown in Figure 11.

Box 3. Sample import command

bulk_buildings.py -f addresses.csv -u admin -p rover -s

192.168.0.13:8000 -c ACSZOLGNfY

For the text right after -c type only the column codes for the columns in your data file; this
is the string of characters you wrote down in step 8. If you used a different filename, give
that filename in -f [your file name], without the quotes or square brackets. If you changed
your administrators username or password, give the correct username and password in -u

13
 [username] -p [password] without quotes or square brackets. The numbers that appear right
after -s are the numeric IP address and port number. For example, if your host IP address is
192.168.0.13 and the port number is 8000, give -s 192.168.0.13:8000 in the following
command.

Figure 11. Entering the import command at the command prompt

If you imported 1 address, you should see 1 (of 1 possible) buildings added, as in Figure
12.

Figure 12. The command prompt after the successful import of 1 building

23. In your browser, in ROVER, click the refresh button in the lists of sites and worksheets
(Figure 13) and check that the building locations now appear in ROVER.

14
Figure 13. ROVER pre-earthquake server site list after successful import

24. Type exit at the Windows command prompt and press Enter, as shown in Figure 14.

Figure 14. How to exit the command prompt

5. Import your Data, Case 2: Using a Service Provider

25. If a service provider is hosting your data for you, log in to their server. Enter your username
and password. For example, in the ROVER testbed at http://testbed.isti.com, the default
username and password are admin and rover, respectively. See Figure 15.

15
Figure 15. Logging into a service provider's ROVER server

26. Click Select FEMA 154 (pre-earthquake), as shown in Figure 16.

Figure 16. ROVER Server system selection

27. Click View list of active sites and their assignments, as shown in Figure 17.

16
Figure 17. Getting the list of sites from the pre-earthquake server main menu

28. The host should have given you a numeric IP address for their server. If not, and you only
have a text address such as testbed.itsi.com, you need to find the numeric IP address. To do
that, in Windows, open the Command Prompt. In Windows 7 for example, touch the Start
button, select All programs, navigate to the Accessories folder, and click Command
prompt, as shown in Figure 18. That will open a window where you can type commands, as
shown in Figure 19.

17
Figure 18. How to find the Windows command prompt in Windows 7

Figure 19. The Windows command prompt in Windows 7

18
29. In the command prompt window, type ping testbed.isti.com without the quotes, and write
down the numeric IP address you see in square brackets after the reply. For example, the
reply might be Pinging testbed.isti.com [74.221.210.95] with 32 bytes of data. In this
example, the numeric IP address is 74.221.210.95. See Figure 20.

Figure 20. How to ping the ROVER server

30. If you have not already opened the command prompt, do that now. In Windows 7 for
example, touch the Start button, select All programs, navigate to the Accessories folder,
and click Command prompt. That will open a little window where you can type the
command prompt. Change directories to the folder where you placed the script and data file
your Desktop by typing CD Desktop\ROVERdata at the command prompt as shown in
Figure 21 and pressing Enter.

Figure 21. How to change to the ROVERdata directory

You should now see this (Figure 22):

19
Figure 22. Command prompt after you have successfully changed directory

31. Type the command shown in Box 4 after the command prompt.
Box 4. Sample import command

bulk_buildings.py -f addresses.csv -u admin -p rover -s

74.221.210.95 -c ACSZOLGNfY

For the text right after -c type only the column codes for the columns in your data file; this
is the string of characters you wrote down in step 8. If you used a different filename, give
that filename in -f [your file name], without the quotes or square brackets. If you changed
your administrators username or password, give the correct username and password in -u
[username] -p [password] without quotes or square brackets. The numbers that appear right
after -s are the numeric IP address. For example, if the service providers numeric IP
address is 74.221.210.95, the number string right after -s is that string, for example -s
74.221.210.95 as shown in Figure 23.

Figure 23. Entering the command to import the building data

If you imported 1 building location, you should now see something like Figure 24:

20
Figure 24. The command prompt after successfully importing the building data

32. In your browser, in ROVER, view the lists of sites and worksheets and check that the
building locations now appear in ROVER. See Figure 25 for an example.

Figure 25. Pre-earthquake server site list after successfully importing building data

33. Type exit at the Windows command prompt as shown in Figure 26 and press Enter.

21
Figure 26. How to exit the Windows command prompt

22