Django Blog Zinnia Limit User to Own Entry/Post

I just need a simple yet flexible django based blog application, and found django-blog-zinnia. I really admire how simple it is. The only major adjustment that I need is to limit user to his/her own blog entry/post.

Combining basic django (admin) knowledge with soome info from django-blog-zinnia documentation, I found a simple solution. Of course this is related with django-blog-zinnia's admin interface, and there is a documentation about how to make some modification to the EntryAdmin. Read more from here.

The idea is to create a django application and create admin.py on that app's top level directory (the same location with models.py).

Here is how my admin.py codes look like:
############# admin.py #################

from django.contrib import admin
from zinnia.models import Entry
from zinnia.admin.entry import EntryAdmin

class FilteredEntryAdmin(EntryAdmin):
    def queryset(self, request):
        qs = super(FilteredEntryAdmin, self).queryset(request)
        if request.user.is_superuser:
            return qs
        return qs.filter(authors=request.user)

admin.site.unregister(Entry)
admin.site.register(Entry, FilteredEntryAdmin)

############# end of admin.py #################

Remember to add the newly created app in settings.py INSTALLED_APPS, and make sure you put it below zinnia.

Django's documentation is quite complete and comprehensive. To get more information about ModelAdmin, queryset, etc, go to this link. From there I got the idea of overriding the queryset and use filter to limit the entry based on user.

(pip) Install Python Imaging Library (PIL)

I had a problem when I tried installing Python Imaging Library (PIL) version 1.1.7, within virtualenv, using pip. Actually, there was not any errors raised during installation, but when I used it in Django for the ImageField, I kept getting 'Upload a valid image. The file you uploaded was either not an image or a corrupted image' error message.
I figured out that the problem is due to the manual installation of PIL using pip, instead of using Debian/Ubuntu's apt-get.
So, the simplest solution would be using the
$ sudo apt-get install python-imaging

The thing is, I need PIL installed in my virtualenv environment. Getting some info from this link, I figured out a way to install PIL within virtualenv. In order to do that, some changes need to be done inside the PIL's setup.py script.
Step 0: sudo apt-get install libjpeg62 libjpeg62-dev
Step 1: get PIL source kit (here).
Step 2: untar, unzip the file
Step 3: locate setup.py from Step 2, and open that file for editing
Step 4: around line 200 (you should see other paths around there), add the following
add_directory(library_dirs, "/usr/lib/i386-linux-gnu")  # for 64 bit replace with /usr/lib/x86_64-linux-gnu
Step 5: tar and gzip the modified PIL source.
Step 6: activate your virtualenv, should you need it
Step 7: pip install

I made some test with my django-blog-zinnia project, by uploading some pictures and no more error messages.

Python SimpleHTTPServer

I needed to do some test and share a file on different machine. Fortunately there is a very simple solution for what I need, which is using Python's SimpleHTTPServer.
What I needed to do is simply go to the directory where the file is and then execute the following command:

$ python -m SimpleHTTPServer

Executing above command without arguments will run the webserver on 0.0.0.0:8000.

For more information just click here.

Interacting with Django Tastypie API Using Python Librarys

If you already have some django web-app, consider extending its functionality with django-tastypie. It has pretty decent docs as well. With django-tastypie you can create RESTful Web Service. I am not going to talk about django-tastypie in this post, instead I would like to share how to interact with django-tastypie api using python's urllib and urllib2 libraries.

The following codes gives some idea about how to get data using the API, with ApiKeyAuthentication method.

import urllib, urllib2

api_url = 'http://localhost:8080/api/app1/model1/?username=apiuser1&api_key=apikey_for_apiuser1'
http_req = urllib2.Request(api_url, headers={'ACCEPT': 'application/json, text/html'})
try:
    resp = urllib2.urlopen(http_req)
    contents = resp.read()
    print contents
except urllib2.HTTPError, error:
    print he.getcode()


The codes below gives some idea about how to create data using the API, with ApiKeyAuthentication method.


import urllib, urllib2
import simplejson as json


api_url = 'http://localhost:8080/api/app1/model1/?username=apiuser1&api_key=apikey_for_apiuser1'
json_post = json.dumps({"model_name": "TEST13"})
http_req = urllib2.Request(api_url, json_post, {'Content-Type': 'application/json', 'ACCEPT': 'application/json, text/html'})
try:
response = urllib2.urlopen(http_req)
print response.read()
print response.getcode()
print response.info()
print response.msg
except urllib2.HTTPError as he:
print he.getcode()
print he.info()
print he.msg
except urllib2.URLError as ue:
print ue

Both examples using json data format. Remember to set the appropriate Content-Type and ACCEPT header data if using different data format.
If it is necessary to use https secure connection and proxy, refer to this post to get more information.

Python HTTPS POST Data Over Proxy

Posting HTTPS POST data over proxy is pretty simple in python. Everything is done mostly with urllib2. The only thing that is done outside urllib2 is when encoding the POST data, which uses urllib.urlencode.
Here is an example:


import urllib, urllib2

PROXY = urllib2.ProxyHandler({'https': 'http://your_proxy:your_proxy_port/'})
PROXY_OPENER = urllib2.build_opener(PROXY)
urllib2.install_opener(PROXY_OPENER)

URL = 'https://www.post-test-site.com/post.php'
POST = {'data1' : 'Your Data1', 'data2' : 'Your Data2'}

POST_DATA = urllib.urlencode(POST)
HTTPS_REQ = urllib2.Request(URL, POST_DATA)
RESP = urllib2.urlopen(HTTPS_REQ)
OUTPUT = RESP.read()

print OUTPUT

When your proxy needs authentication, then you need to use:
urllib2.HTTPBasicAuthHandler and/or urllib2.HTTPPasswordMgrWithDefaultRealm

Python ValueError Bad Marshal Data

I encountered this problem when I copied my django project along with (cherrypy's) wsgiserver and started it.

To solve this issue, I needed to remove wsgiserver.pyc, and re-run the program. Just like that.

So, it is a good idea to clean all the *.pyc's, when moving/copying codes in a different computer, and let python recreated them all.

Python Idioms, Idiomatic Python

Get to know more about Python Idioms.

Just some samples of nice python idioms:

result = 'ok' if x > 10 else 'not ok'
result = [ 3 * d.Count for d in data if d.Count > 4 ]

Here are some links which I found very interesting:
http://python.net/~goodger/projects/pycon/2007/idiomatic/handout.html
http://bayes.colorado.edu/PythonIdioms.html

I will find more links and add them to this post later on.

Django + Eclipse + PyDev + Aptana

For those who likes Django, which IDE do you like the most for development? Of course this is subjective thing, and it's always interesting to see different answers to that question. Some people says vim, notepad++, some other likes commercial IDEs.
In this case, I prefer eclipse with pydev and aptana plugins.

Once you have Python, Django, and Eclipse installed in your system, it is time to install the necessary plugins.

First is to install PyDev plugin. Install new software in eclipse using this url http://pydev.org/updates.
Then of course configure PyDev to use the appropriate python interpreter that you already have. To do that go to eclipse preferences.

Next is to install Aptana Studio, using this url http://update.aptana.com/install/studio.

Now, you can create django project, either using eclipse's wizard, or using django-admin.py script and import it in your eclipse project.
If you use django orm you will notice that 'DoesNotExist' keyword is not recognized. To solve this matter, do the following:
go to eclipse preferences -> pydev -> editor -> code analysis -> Consider the following names as globals --> add 'DoesNotExist'

Then it is time to configure your project so you can run and debug it in eclipse.
To do that, you need to perform the following:
go to project properties -> run/debug settings -> new (python run)
In main tab:
set the project, choose by clicking the browse button.
set main module, choose by clicking the browse button and select the project's manage.py. When you are doen, it would show something like the following:
${workspace_loc:project/app/path/manage.py}

This step is important, if you are using Aptana Studio plugin. By default django will user port 8000 for it's web server, which conflicts with Aptana Studio plugin service. So it is better to use some other port for django web server.

Again, go to project properties -> run/debug settings.
In arguments tab:
runserver 8080 --noreload

Notice that I am using port 8080 for the web server.

In environment tab:
add new variable:
variable: DJANGO_SETTINGS_MODULE
value: settings

At this point you can test running your django project by using the run setting that configured previously. Enjoy debugging your django project in eclipse.

Python and CSS Selector

Python has a lot of awesome modules for working with html documents. My favorite one is lxml modules. Last time I wrote about xpath using that module.
Now, it is time for css selector using lxml.cssselect.

Straight to an example of using it:

from lxml.cssselect import CSSSelector
css_sel = CSSSelector('table.tableSchedule tr')
html_doc = lxml.html.fromstring(some_web_html_string)
row_els = css_sel(html_doc)
print row_els

Sweet and simple.

I am still playing around with it and found that using direct descendant css code is not possible. Something like the following:

css_sel = CSSSelector('table.tableSchedule > tobdy > tr > td')

Please leave a comment if you are able to use ">" in the css code part.

django deep serializers

Django has it's own very nice serialization module. A very useful tool to serialize models in django projects. There is limitation to it, though.
When one serialize a model (table) which has relationship (ForeignKey) to other model, then you will only get the foreign_key, instead the whole object. Something like the following:

[ {
    "pk": 1,
    "model": "store.book",
    "fields": {
        "name": "Mostly Harmless",
        "author": 42
    }
} ]

In some cases, deep serializing is often necessary. Now, let me introduce you to this magnificent python module which extends django's built-in serializers. DjangoFullSerializers.
With this, you can get something like the following:

[ {
    "pk": 1,
    "model": "store.book",
    "fields": {
        "name": "Mostly Harmless",
        "author": [ {
             "pk": 42,
             "model": "store.author",
             "fields": {
                  "name": "John Doe",
                  "title": "Dr."
             }
        } ]
'    }
} ]

I have used it and really satisfied with the result. Many thanks to the developer.

Python and RS323 (ESC/POS) Receipt Printer

Currently, I am working a bit with RS323 (ESC/POS) Receipt Printer. The most interesting part working with it is I do not need to have a driver. This is very helpful because I am planning to use it with Linux OS.
I decided to use Python, so anytime I need to switch to Windows OS, nothing need to be changed in the code part.

Since the printer is using RS323 connection, I need to find the simplest way of communicating with that device in Python. With no effort I found pySerial. Taken from the website itself: This module encapsulates the access for the serial port. It provides backends for Python running on Windows, Linux, BSD (possibly any POSIX compliant system), Jython and IronPython (.NET and Mono). The module named "serial" automatically selects the appropriate backend.

After installing pySerial, I started to play around with it. It is really simple and straight-forward.
It took me sometimes to understand ESC/POS commands, since I was totally not familiar with it.

Just to give some idea on how simple it is, here is an example:

import serial
ser = serial.Serial(port='/dev/ttyS0', baudrate=19200)
ser.portstr   #check which serial port
ser.write('Hello World!\n')
ser.close()

I spent sometimes trying to figure out how to work with ESC/POS. One of the issue is the baud rate setting. This must be configured properly otherwise the printer will not print the data properly.

Another thing is that the printer will not print anything if you don't tell it to print data in it's buffer. One way to do this is terminate any string/data you want to print with '\n' (or ASCII decimal 10).

Next step is to know what ESC/POS commands available for your type of printer. I downloaded some docs from the vendor's website, but unfortunately there are only minimum info about ESC/POS programming.
Fortunately, ESC/POS commands is pretty much a standard, so I found some other documents from different vendor, by googling it, and got more informations.

Another interesting thing is that when I tried to print several lines of data in Linux, somehow only partial string in certain line that was printed. Got me confused for a bit, then I realized that this is printer buffer issue. The buffer in the printer is too small for the amount and speed of the data I sent to it. In this case adjustment need to be done, so the printer buffer won't get overloaded and discarded the incoming data.

I don't really like current receipt printer that I work on. Minimum documentations, vendor specific power supply cable, and some other stuff.
I haven't search much about other receipt printer from other vendors, but I am pretty sure that there are some that are more portable to work with.

Using django-admin.py within cygwin

Sometimes I still work in windows environment. I always have cygwin installed in windows environment since I use a lot of *nix tools and scripts.
Currently, I am working a lot with python and django, very nice scripting language and powerful web framework. I used native windows python installation, instead of installing it within cygwin.
The problem occur when I tried to invoke django-admin.py script in cygwin's bash shell. It won't run. It is due to windows version of python expecting windows path of the django-admin.py script as the argument.
Here is the exact situation. Windows python expecting, something like this:

python c:\Python26\Scripts\django-admin.py

but since I run that command within cygwin, it will become like the following:

python /cygdrive/c/Python26/Scripts/django-admin.py

and throws some error messages.

So, I searched a bit and found out that cygwin has a tool that converts *nix path to windows path. The name of the tool is cygpath.

And here is what I did to solve the problem.
I created the following file:
/usr/local/bin/django-admin.py

which contains:
python.exe "$(cygpath -aw "/cygdrive/c/Python26/Scripts/django-admin.py")"

Of course you need to make /usr/local/bin/django-admin.py executable

chmod +x /usr/local/bin/django-admin.py

and you can run django-admin.py from anywhere.


Check out the following links for more information:
http://www.cygwin.com/cygwin-ug-net/using-effectively.html
http://www.cygwin.com/cygwin-ug-net/using-utils.html#cygpath

XML Document processing comparison between java and python

I had a project that requires xml document processing. At first, I was thinking to do it using Java, but since the project was not so big, I decided to use Python. I really have to say that it was a good decision. In my opinion, xml doc processing feels more natural in Python compare to Java.

Let's go straight to the code comparison.
There are many steps involved in java before actually getting the xml doc elements that match with xpath expression defined.

DocumentBuilderFactory factory = DocumentBuilderFactory.newInstance();
factory.setNamespaceAware(true); // never forget this!
DocumentBuilder builder = factory.newDocumentBuilder();
Document doc = builder.parse("books.xml");

XPathFactory factory = XPathFactory.newInstance();
XPath xpath = factory.newXPath();
XPathExpression expr = xpath.compile("//book[author='Neal Stephenson']/title/text()");

Object result = expr.evaluate(doc, XPathConstants.NODESET);

NodeList nodes = (NodeList) result;
for (int i = 0; i < nodes.getLength(); i++) {
    System.out.println(nodes.item(i).getNodeValue()); 
}

I took the example from IBM website, here. You should check out the link if you are interested using xpath in java.

Now, in Python, using lxml library.

doc = etree.parse("books.xml")
nodes = doc.xpath("//book[author='Neal Stephenson']/title/text()")
print(nodes)

I mostly use java for my projects, pretty much happy with it, even though sometimes I wish that java has some syntactic sugars like in C#.
So, if you have the option to choose which programming language for your xml projects, give Python a try. I am sure you will like it.

Processing xml document in python using lxml

Processing xml document in python using lxml

In previous entry, I discussed about libxml2. Now, I am going to discuss about lxml, which in my opinion is awesome. I really appreciate the developer team that create lxml. It makes processing xml document easier.
The best way to get it by going to this link and choose the appropriate version. As soon as you installed lxml, try to use it. Here is a sample of how to use lxml.

from lxml import etree
import sys

def getChildInfo(child):
 return "id: " + child.get("id") + ", data: " + child.text + "\n"

xmlString = "c1c2"
tree = etree.fromstring(xmlString)
children = tree.xpath("//child")
sys.stdout.write(str(len(children)) + "\n")
for el in [ getChildInfo(child) for child in children ]:
 sys.stdout.write(el)

childList = tree.xpath("//child[@id='1']")
sys.stdout.write(getChildInfo(childList[0]))
child = tree.xpath("//child[@id='2']")
sys.stdout.write(getChildInfo(childList[0]))

For more examples and other information about lxml, check out this link.

Processing xml document in python using libxml

Processing xml document in python using libxml

I had to spent sometimes to gather information about processing xml document in python. There are several different libraries to use, such as minidom, libxml, etree, etc. During my work, I had been working with xpath, so I need xpath support in the built-in library that I work with, so I chose libxml. It is difficult for me to gather information on how to use that API. There are minimum documentations, guides, examples, and tutorials out there.
After spending sometimes, I became familiar on using it. I am writing this, so other people could simply learn on how to use libxml in more realistic situation. Many examples I found in the internet is way too basic, and does not suit my need.


example 1:
--------------------------------------

import sys
import libxml2

def main():
 doc = libxml2.parseFile("data.xml")
 ctxt = doc.xpathNewContext()
 # get student elements in xml doc
 res = ctxt.xpathEval("//student")
 # output all properties for "student" element using xpath
 sys.stdout.write(res[0].get_properties().content + "\n")

 for chld in res[0].children:
     if chld.type == "element":
             # output all information within the child element
             sys.stdout.write(chld.content + "\n")

 doc.freeDoc()
 ctxt.xpathFreeContext()

if __name__ == '__main__':
 main()

--------------------------------------


example 2:
--------------------------------------

import sys
import libxml2

def main():
 doc = libxml2.parseFile("layout.xml")
 ctxt = doc.xpathNewContext()
 # get all "property" element that has attribute "Location",
 # which parent is "object" element that has attribute "pickList*"
 res = ctxt.xpathEval("//Object[starts-with(@name,'pickList')]/Property[@name='Location']")
 for r in res:
     sys.stdout.write(r.content + "\n")

 sys.stdout.write(str(len(res)))

 doc.freeDoc()
 ctxt.xpathFreeContext()

if __name__ == '__main__':
 main()

--------------------------------------


It is also useful to see the source code of libxml.py, just to see what's going on under the hood.

Not long after writing this post, I found even better xml API, which is lxml. I will make separate post about this wonderful xml API for python.