Wednesday, October 14, 2009

Encoded Polylines for Static Google Maps


The Google Static Map has a size limitation of 2400 characters which is because of IE's url size limit. Since the static map is created by sending the polyline endpoints through the URL, it becomes easy to reach that limit with a curvey path on the map...for example...gpx paths for hiking paths. Using the ecoded polyline feature, which calculates the path segments by their relative changes, and converting to a more efficient binary representation, the data set can be greatly compressed in the URL.


The Urban Sensing Wiki at UCLA about "How-to use Google Maps polyline encoding to compact data size" and gives the following Python functions to use.




# Encode a signed number into the google maps polyline encode format.
# from: http://wiki.urban.cens.ucla.edu
def encodeSignedNumber(num):
sgn_num = num << sgn_num =" ~(sgn_num)" encodestring = "">= 0x20:
encodeString += chr((0x20 | (num & 0x1f)) + 63)
num >>= 5
encodeString += chr(num + 63)
return encodeString

# Create the encoded polyline and level strings.
# @points: [{'lat':100, 'lng':50}, {'lat':102, 'lng':49.8}]
# This function takes a list of dictionaries (each dictionary contains a
# single 'lat','lng' pair) and returns a dictionary:
# {'encoded_points': string, 'encoded_levels': string}.
# from: http://wiki.urban.cens.ucla.edu
def encodePolyPts(points):
i = 0
plat = 0
plng = 0
encoded_points = ""
encoded_levels = ""
for i in range(0, len(points)):
point = points[i]
lat = point['lat']
lng = point['lng']
level = 3 - i%4
late5 = int(math.floor(lat * 1e5))
lnge5 = int(math.floor(lng * 1e5))
dlat = late5 - plat
dlng = lnge5 - plng
plat = late5
plng = lnge5

encoded_points += encodeSignedNumber(dlat) + encodeSignedNumber(dlng)
encoded_levels += encodeNumber(level)

return {'encoded_points': encoded_points, 'encoded_levels': encoded_levels}


To use this encoding capability to draw a static Google map, the (points) list needs to be built. The variable is
expected to be a list of dictionaries. The documentation shows us it is expecting the format to be as follows:


@points: [{'lat':100, 'lng':50}, {'lat':102, 'lng':49.8}]


The following function retrieves the datastore item and parses the lat and lng values
to create a single dictionary which is then appended to the list.


Note: in this example the "newStr" variable is a JSON string from the datastore which is loaded into a python object using the
simplejson utility. It can then be iterated to find all the stored lat and lng points.



def mapShow(request, route_key):
"""creates a static google map of the entire route """
"""written by Thea Ganoe, October 2009"""
#http://maps.google.com/maps/api/staticmap?size=400x400&path=weight:3|color:orange|enc:polyline_data
user = users.get_current_user()
route = models.Route.get(route_key)
pathList = []
newStr = route.way
#'{"description":"'+ route.routeName +'", "img":"'+route.img+'", "getPic": "' + route.img +'", "zoom":"'+ zoomstring +'" , "longitude":"'+ route.longitude +'", "latitude":"'+ route.latitude +'", "position": "0", "heading": "" }'
ways_obj = simplejson.loads(newStr)
sizeofWay = len(ways_obj['way'])
for way in ways_obj['way']:
pathDic = { 'lat': float(way['latitude']), 'lng': float( way['longitude']) }
pathList.append(pathDic)
# points: [{'lat':100, 'lng':50}, {'lat':102, 'lng':49.8}]
polyPts = encodePolyPts(pathList)
return respond(request, user, 'staticMap', {'enc':polyPts } )


Since the Django template system is used by this Google App Engine application, the return value will merge the staticMap.html template with the server response. To use the enc value, it is necessary to specify which list is needed, enc.encoded_points or enc.encoded_levels. This template looks like this:


<image src="http://maps.google.com/maps/api/staticmap?size=512x512&path=weight:2|color:red|enc:{{ enc.encoded_points }}&sensor=false&key=myveryownbiglongkeyfromGoogle"/>



To create a static map, the levels data is not needed. If a live Google Map was being displayed, the levels values would be used by it, because the enc list contains dictionaries of . The above function, Showmap() will return the data needed for either type of map. This is because the return value enc actually contains list of points and a list of levels.

'encoded_points': encoded_points, 'encoded_levels': encoded_levels

Tuesday, May 26, 2009

Dynamic HTML SELECT forms for AppEngine

AppEngine
dynamically created HTML select forms for Django python templates

I didn't see this particular solution anywhere so I'll post it here, since it something that may come up for anyone trying to do a XMLHttpRequest() in Google AppEngine. Normally I use the Django forms in App Engine but I wanted to quickly add some query filters without having to create a model for them.
This could happen when the web app needs to interface with the user. In this example, I simply wanted to filter the dataset by the location. So I have one select box with all the states listed. I do have a Django model but it is used for creating new records, not filtering lists.

First step, put the forms in the html template (this uses built-in Django template tags ):


<form method="post" id="frmSearchBox">

<table>
<tr><td>
<SELECT name="states" id="states" onChange="selectState(this)">
{% for stt in states %}
<option name="op" value="{{ stt }}" >

{% endfor %}
</SELECT>
</td>
<table>
<input type="button" name="btnSearch" value="Submit" onclick="javascript:searchStates();">
<form>


Notice this form is a post. Because I want to add AJAX to only update a portion of the page.

function searchRoutes() {
//the submit button on the searchbox form triggers this function
//this will do an ajax post to the server and filter the routes according to the user selection

var states = document.getElementById('states')[document.getElementById('states').selectedIndex].innerHTML;

var body ="states=" + states ;
var req = createXMLHttpRequest();
req.open('POST', '/rpcRouteSearch/', true);
req.setRequestHeader("Content-type", "application/x-www-form-urlencoded");
req.setRequestHeader("Content-length", body.length);
req.setRequestHeader("Connection", "close");
req.onreadystatechange = onAddSuccess;
req.send(body);

function onAddSuccess(response) {
//alert("in searchRoutes callback: " + req.readyState);
//replaces the #_nodes with the slideshow
if (req.readyState != 4) {return; }
var serverResponse = req.responseText;
var mg = document.getElementById("leftImages");
mg.innerHTML = serverResponse;

}
}



================
Now a peek at the python in views.py, our callback is waiting for the line that starts with request.method == 'POST':


def intro(request):
user = users.get_current_user()
statelist = ['All']
if request.method == 'GET':
routes = db.GqlQuery("SELECT * FROM Route" )
numroutes = routes.count()
"""Next sections prepare data for the search box forms """
for route in routes:
statelist.append(route.state)
"""this next trick removes duplicate values! """
statelist = list(set(statelist))
return respond(request, user, 'intro', {'routes' : routes, 'numroutes' : numroutes, 'states' : statelist })
if request.method == 'POST':
"""AJAX call, use post when filtering with the search box form """
post= request.POST.copy()
statefilter = post['states']
statefilter = statefilter.strip()
# The Query interface prepares a query using instance methods.
q = db.Query(models.Route)
"""q = models.Route.all()"""
q.filter('state = ',statefilter)
# The query is not executed until results are accessed.
routes = q.fetch(25)
numroutes = q.count()
statelist = ['All']
return respond(request, user, 'introRoutes', {'routes' : routes, 'numroutes' : numroutes, 'states' : statelist})


====

The lines that were nonstandard, are these

statefilter = post['states']
statefilter = statefilter.strip()

It was necessary to use the string function strip to get the results to work properly in the filter function. All it does is remove whitespace from the ends of the string (not inside) and now the variable is ready to be used by the filter.

======

The python code is going to use a template called "introRoutes.html" to build it's serverResponse that gets sent to the waiting Callback function in the javascript.

Since we are working with an HTML form select box, it would be nice to set the SELECTED value, to make sure the form is reset for the user, moreso important when there are many of these select options...otherwise the user has to reset them all manually. We want to set the form back to 'All' after every use, the 'All' setting is added at the beginning of the Python function,

statelist = ['All']

and then the other states available are added to it. To make sure ALL is always SELECTED, the attribute must be written to the form, in this case we are in an asychronous call to the server, but it works the same way.

Here is an extra $2 trick..use Django’s built-in template tags and filters! Anything in the {% isPython %} commands and the bracket tags {{ areVariables }}. These are more fully described at http://docs.djangoproject.com/en/dev/ref/templates/builtins/#ref-templates-builtins

Example of dynamically created HTML select forms for Django python templates.


<em>
<select name="states" id="states" style="width: 6em;"> <option name="op" value="{{ stt }}" ifequal="" stt="" all="" selected="selected" endifequal="">{{ stt }} {% endfor %} </option></select>
</table></form></em>

Sunday, February 15, 2009

Configuring Ubuntu, Eclipse w/phpEclipse plugin, and LAMPP running on a web server

Developing with Eclipse. I've always been a notepad IDE person (obviously hence the blog name but....real work has required me to use Eclipse and CVS).

This covers Ubuntu/8.04 (hardy), Eclipse Ganymede with phpEclipse plugin, and LAMPP. LAMPP is a quick way to get an Apache2 web server up and running for development purposes, with mySQL and PHP automatically set up. But the ultimate solution and purpose of this post, getting Eclipse to output files that run on a web server, will work on any web server. In addition configure phpEclipse and Eclipse workspace folders which requires a few additional steps which are detailed here.

If you don't have LAMPP, you can get xampp-linux-1.7.tar.gz from:
Linux http://www.apachefriends.org/en/xampp-linux.html

Note: If apache2 is installed somewhere else on the system, LAMPP can coexist, but the other Apache daemons will have to be stopped before LAMPP will start (system will check and warn).

Eclipse Ganymede was downloaded and installed from the Eclipse.org site since an older Eclipse was available from the Syntaptic Package Manager and I wanted Ganymede, but really you can use whichever you want, it will not affect these instructions.

Once LAMPP is running and localhost is showing the XAMPP page and Eclipse is configured and loaded with phpEclipse and all the web tools, the Eclipse workspace will normally be placed in your home folder. However, if you're writing PHP files with Eclipse (you installed phpEclipse, afterall) obviously both the web server and Eclipse needs to know where to find the folders. The secret is to put those Eclipse php folders on the webserver. Then put symlinks in the Eclipse workspace that point to the those folders.

Demonstration:

1. make a folder called 'projectX', or whatever you want to call it, in htdocs (the default server document root which can be found at /opt/lampp ) and give it permissions so it is owned by the user:

chown $user /opt/lampp/htdocs/projectX

(in all these examples, $user should be replaced by your Ubuntu username in the commands)

2. make a symbolic link to this to file, and put it (the symlink) in the workspace to be used by Eclipse to hold projects. When Eclipse saves to the folder, the files will actually be written to the folder that was created in the web server's document root (in htdocs).

The rule for creating a symbolic link:

ln -s /path/to/real/file /path/to/non-existant/file

modified for this demo:

ln -s /opt/lampp/htdocs/projectX /home/$user/workspace

Which will result in a folder with an arrow on it in the users eclipse workspace!! It will have zero items in it until files are download from Eclipse CVS or the phpEclipse IDE is used to create or edit files. Most importantly, the files can be browsed and read by the development web server.

Friday, January 16, 2009

Python Preview

In order to do much with Google App Engine, Python server programming awaits with its strange mix of being an old programmer's program and being of the future with a direction that will carry us away from everything as we've known it. Python is different from programs like PHP or Perl or ASP because it isn't a scripting language. It's a real meat and potatoes programming language that can go toe to toe (maybe, almost) with the likes of C++. To work with Python, server side developers will spend the most time figuring out how to put data in a list or a dictionary or some combination of the two. The Python book is 5 inches thick but I spend most of my time around page 20, preview about lists and dictionaries because to get my data processed, this was the only way. (My data being strings of json-like recordsets)

And there is a good reason for that. It is not necessary to name fields or columns or components. Python doesn't care. Just stuff the data into a lists, put the list in a dictionary and follow a few rules to iterate through it. It's not even necessary to declare the data types. Python does that invisibly at run time based on what it finds in the containers.

It works beautifully with Google's Big Table. Formal database records with their keys and nomalized relations are out the window. Goodbye to recordsets and table joins. Who needs a table schema? The process of defining the list and dictionary in Python creates dynamic objects. Which are all any reasonable server needs to manipulate the data before sending it off to any of its possible destinations.