Sunday, December 21, 2008

Reading Visual Basic Binary encoding for screen coordinates

Reading Visual Basic Binary encoding for screen positions, X and y.



This is a slight departure from Ajax-related topics, but since I've been heavily involved with transforming older binary files to modern XML, there was one challenge that took some unexpected calculating that I want to document here.



The older binary files were created by Visual Basic 6, apparently it writes the screen coordinate positions in big endian formats.



In my binary files, I knew these eight bytes contain the screen position and size of a block. Another way to describe this would be a the top left corner and lower right corner, the coordinates (x1, y1) and (x2, y2).



Here are some bytes that describe 4 boxes in a row, with varying widths, but all with the same height,effectively making a table.



4 rows of 8 bytes in hex code format:



00 8C 00 08 01 31 00 69

00 8C 00 69 01 31 00 C5

00 8C 01 C4 01 31 02 7F

00 8C 02 7F 01 31 03 16







In each row, the first two bytes are x1. Next two are y1. Then x2, and last is y2. Little endian doesn't work, the numbers get too big. Using big endian:



ypos1, xpos1, ypos2, xpos2



140, 8, 305, 105

140, 105, 305, 453

140, 452, 305, 639

140, 639, 305, 790





Since we know the y values are the same, column 1 would be y1 and column 3 would be y2.

This would leave column 2 as x1 and column 4 as x2. This works since each row starts where the previous row left off. Note that to find the width, just subtract xpos1 from xpos2 and to find the height, subtract ypos1 from ypos2.



The only reason this was any problem, is that the Intel machines are little endian, the built-in readUint16() gave results that were way too high to be screen positions. Therefore, an extra processing function was needed to make the calculation.



Here is an example from the internet (devx.com) showing a conversion between little endian and big endian.





int SHORT_little_endian_TO_big_endian(int i)

{

return ((i>>8)&0xff)+((i << 8)&0xff00);

}






Here was my own coded solution that reads one byte at a time while parsing through the binary file:



highb = inf.ReadByte()

lowb = inf.ReadByte()

highb = (highb * &H100&)

ypos1 = highb + lowb

Thursday, October 9, 2008

sharing JSON between web page and server

I've created an App Engine application that uses a field in it's model to store a JSON. This application uses Django and simpleJSON. I wrote my own Javascript framework for App Engine, which I'll try to publish so people can use it. I looked at Prototype and some others but wanted more control since I don't think the browser DOM's are all that hard for simple ajax effects and requests. However, if I must use drag and drop, I will probably enlist the help of a framework. So far I have avoided drag and drop.

The place I want to start out is with JSON. My application is like a project that plays the movie and the movie is the JSON, it is the data.

I started with the example from "Google Maps Hacks" By Rich Gibson, Schuyler Erle

Which was a good placeto start dealing with the Google Maps API and building HTML files that run it. The book is several years old already, so I had to update to the Version 2 maps.

An example of an active Javascript Object on a page with a google map can be seen at igargoyle.com and then view the source for the myObj object.


var myObj = {"photos": [
{"img": "http://igargoyle.com/theman.jpg",
"longitude": "-119.236402821", "latitude": "40.754786911",
"title": "Black Rock City: 2005",
"description": "A Playa Slideshow using Google Maps. Just email this to your friends, and when it loads, it will start playing automatically."},...]}



This is an example of a JSON object being used in HTML script. Notice that it is an object named myObj which contains and array named photos. To retrieve the longitude of the ith element in the array (only the first element is shown above) use


longitude = myObj.photos[i].longitude;


If you have a script to run in the browser or any kind of a states to chose between, if the data is in an array index data structure, each item can be accessed directly by it's index. Arrays are very fast for these sorts of finds, where the number is known. Arrays are not so good if searching for a key needs to be done. If your data fits an array, it will work good as a Javscript object. Think of this object as something that is loaded into the client browser memory when the html page loads. It is then ready to be used to update the page display on user requests and/or timing events. This memory object really only needs the subset of information that the given page may require. No need to put everything from your database on the server in the page. Manipulation can be done in the server side code to send only the data needed by the page. This keeps it small, fast and efficient.

Now that there is a Javascript object in memory on the web page, the web page could have some editing features, to update/change the object. It then needs to be POSTed back to the App Engine server and saved in the datastore. To get the object out of memory, just write it to a hidden (or not hidden) form!

here is a javaScript function that does just that:


function myObj2string() {
var newString =JSON.stringify(myObj);
document.getElementById('JSONbox').value = newString;
}


the JSONbox element should be in the body:



Now that we have captured a Javascript object in a form box, it can be POSTED to the server and stored in Big Table.
This is in the python code



modelEntity.way = request.POST['JSONbox']
db.put(modelEntity)




When the model was built (model.py) for modelEntity, a field for the JSON was included as a TextProperty() which has so far worked well for me.

way = db.TextProperty()


Now that JSONs can be created and stored, to retrieve it and use it, follow these steps.

in django views (python server code):

def play(request, modelEntity_key):
...
newStr = modelEntity.way
...
return respond(request, user, 'viewer', {'ways':newStr})



notice the resonse will call the viewer.html template and pass a context containing newStr which will be called ways. Django passes data to the templates by giving it a context. A context is simply a set of variables and their associated values. A template uses this to populate its variable tags and evaluate its block tags.

putting JSON back into an html page via a Django template:


var myObj = {{ ways }} ;



If the model instance has been passed to the template (eg a context like this: {'modelName':modelEntity} ) the JSON can be pulled directly out of it's model and rendered in the HTML (again this is the Django template):



var myObj = {{ modelName.way }} ;



That should illustrate how easy it is to use JSON. It can be stringified and handled like a string on the server for storage purposes and then used by Django's template and context system where it is rendered in html and then loaded as an object into client memory!