Python: Why The global Statement

Often, you come across Python developers debugging a piece of code involving global variables. Sure, global variables are the devil, but when you inherit code for a previous developer, you are just going to have to live with it until there's a budget for refactoring or rewriting it. Typically, if you are referencing a global variable within a function, you would 'import' the global variable into the function scope using the global keyword like this:

film_name = "painted skin"
def film():
  global film_name
  print(film_name)
film()

Now, even when you leave out the global statement, Python is smart enough to figure out that the variable is a global variable, as in this case:

film_name = "painted skin"
def film():
  print(film_name)
film()

Suppose you perform a variable assignment within this second case, you would have code like this:

film_name = "painted skin"
def film():
  film_name="final fantasy"
  print(film_name)
film()

When you execute it, all would seem fine and dandy till you decide that you want assignment statement to occur after the print statement like this:

film_name = "painted skin"
def film():
  print(film_name)
  film_name="final fantasy"
film()

The error that you would now receive reads the following:

UnboundLocalError: local variable 'film_name' referenced before assignment

What has happened in this case and the previous is that when a variable assignment occurs, Python creates a local variable instead of using the global variable. In the case where we only printed the variable value, the global variable was referenced. This can be resolved using the global statement, as indicated in the first case.

Python 2 and Python 3: Two 'different' platforms

When you decide to do any Python development today, the first choice that you come across is the version of Python that you want to use. There's Python 2, the programming platform with lots of libraries that do everything except defying gravity, and then there's Python 3. Python 3 is really good as a language, but we have yet to see the programming libraries and support from the larger community. There are tools to assist in porting Python 2 code to Python 3, and vice-versa but there just has not been enough interest generated within the community to create the everything-but-the-kitchen-sink feeling among Python 3 users so the 'early adopters' (c'mon, it's been a few years since Python 3 was introduced, it isn't early any more!) don't really get all of the niceties that Python 2 users get.

The one area where you would notice a shortfall of Python 3 is in GUI application development. Most GUI libraries/frameworks are in a beta or beta-like stage and using them is like having a swinging pendulum-blade right over you. For web development, there are frameworks supporting Python 3 so if your framework of choice does not support Python 3 yet, you can simply switch over to another framework that does.

If you are a newbie who wants to learn Python, there are a couple of built-in Python modules that have been renamed so you can't follow all the examples that you see online if all you have is a Python 3 environment (or vice-versa, in which case you have a Python 3 example but all you have is Python 2 installed). I have been tinkering with Python every now and then, and when I decided to actually take the plunge and adopt Python 3 for automating some of the application configuration that I occasionally have to do, I started off with Python 3. Initially, figuring out the differences in the names of the modules took a while. Now, I don't even look back at Python 2 and hope everyone else becomes a part of the shift from Python 2 to Python 3.

Enabling AutoComplete In ADT

I got the Android Development Toolkit, which includes Eclipse and the SDK for developing Android applications. After having a go at it for a couple of minutes, I noticed that I didn't have any autocomplete. Being a newbie at Android development (no experience with Android development, apart from a few days of Android coding in 2010 or 2011), I really needed the autocomplete to work so I looked for something that could point me the right way. I finally stumbled upon the settings at:
Window > Preferences > Java > Editor > Content Assist > Advanced

I enabled the Java Proposals, Java Type Proposals, and Java Non-Type Proposals, and as soon as I dismissed the dialog box by clicking the Ok button, I was able to use Ctrl+Space for the code autocompletion!

Python: Subclassing the BaseHTTPRequestHandler

As discussed in my previous post, Python provides the HTTPServer class for listening to HTTP requests and dispatching them to request handlers. To write our own handler, we can create a subclass of BaseHTTPRequestHandler and handle the requests by providing functions named with the 'do_' prefix followed by the HTTP action verb (Eg. do_GET, do_HEAD, do_POST etc). The following is an example that uses this approach to create a web server.

from http.server import HTTPServer
from http.server import BaseHTTPRequestHandler
from http.server import SimpleHTTPRequestHandler

def run(server_class=HTTPServer, handler_class=BaseHTTPRequestHandler):
 server_address = ('', 8000)
 httpd = server_class(server_address, handler_class)
 httpd.serve_forever()

class MyHandlerForHTTP(BaseHTTPRequestHandler):
 def do_GET(self):
  self.send_response(200)
  self.send_header('Content-Type', 'text/plain')
  self.end_headers()
  self.wfile.write(bytes('Hello World\n', 'UTF-8'))
  self.wfile.write(bytes('You have requested '+self.path+'\n', 'UTF-8'))

run(handler_class=MyHandlerForHTTP)

In this example, we created a class named MyHandlerForHTTP that inherits from BaseHTTPRequestHandler and defined the do_GET method within it. The BaseHTTPRequestHandler class provides methods to set the HTTP response status code, output the headers, and write data to the response stream. Note that you do not have to call the close method on the wfile member as the BaseHTTPRequestHandler class implicitly calls the flush method of the wfile member after the execution of the do_* method.

Python HTTP Web Server

Python makes it extremely simple to create a web server. I adapted the run function provided in the Python documentation to create the following example.

from http.server import HTTPServer
from http.server import BaseHTTPRequestHandler
from http.server import SimpleHTTPRequestHandler

def run(server_class=HTTPServer, handler_class=BaseHTTPRequestHandler):
 server_address = ('', 8000)
 httpd = server_class(server_address, handler_class)
 httpd.serve_forever()

run(handler_class=SimpleHTTPRequestHandler)

In the code above, we first import the HTTPServer, BaseHTTPRequestHandler, and SimpleHTTPRequestHandler classes from the http.server module (if you want to know more about modules, look at my slides on Python Modules). We then have a run function that accepts the name of an HTTP server class and an HTTP request handler class as parameters.

The HTTPServer class is used to listen for web requests and dispatch them to request handlers. Request handlers are what we use to figure out what needs to be done and to send a response. The BaseHTTPRequestHandler calls methods with a 'do_' prefix followed by the name of the HTTP action verb (more on that in another post). In the example above, we use a SimpleHTTPRequestHandler that provides the functionality out-of-the-box to serve files from the current directory.

Our example above acts as a glue between the HTTP server and the SimpleHTTPRequestHandler class without doing the actual processing on its own. In another blog post, I will describe how you can create your own request handler class to process requests in Python code.

[Python] TypeError: str does not support buffer interface

When writing data, such as to sockets, you probably see a lot of code that calls a write function and passes in a string like this:

self.wfile.write('Hello World')

This code ran perfectly well in Python 2.x, but not so in Python 3.x. In Python 2.x, strings that contained only ASCII characters could be passed as data but if the strings contained characters that couldn't be represented as ASCII characters, a UnicodeDecodeError would occur.

Python 3.x treats text and binary data as distinct types, and raises a TypeError if you provided text where binary data was expected. In Python 3.x, you have to convert text data to binary data using the bytes function, so expect to rewrite your code to look like the following:

self.wfile.write(bytes('Hello World', 'UTF-8'))

SQL Monitoring / Query Analysis with MySQL

I'll keep this post a short one. I wanted to view the queries that are being run against a MySQL database and found the simplest way to do it is to view the queries in the general log file using tail. There are 2 variables: general_log to set the logging ON or OFF, and the general_log_file, which indicates the location of the log file.

The following is dump of my MySQL client output:

mysql> SHOW VARIABLES LIKE "general_log%";
+------------------+----------------------------------+
| Variable_name    | Value                            |
+------------------+----------------------------------+
| general_log      | OFF                              |
| general_log_file | \xampp\mysql\data\reddythink.log |
+------------------+----------------------------------+
2 rows in set (0.00 sec)

mysql> SET GLOBAL general_log = 'ON';
Query OK, 0 rows affected (0.08 sec)

mysql> SET GLOBAL general_log = 'OFF';
Query OK, 0 rows affected (0.04 sec)

Oracle DECODE and CASE expressions

The use of the DECODE function in Oracle is to substitute a value or expression with another. The CASE expressions serve a similar purpose but can be used in a wider variety of scenarios.

Let's say you have a character field in the database and you want to select data from the table and display a more user-friendly value instead of a single character. You can use the DECODE function to specify the column or expression to be checked followed by a pair of values or expressions to be matched with a substitution value, and finally a substitution value to be used if none of the matching values or expressions matched. Here is an example:

A table r5translines containing a column trl_type has the value 'I' for goods issued (sent out), 'RETN' for goods returned (such as when a defective product is returned), and 'RECV' for goods received (such as goods received from a supplier). To display this information in a user-friendly form, the SELECT statement can be writted with a DECODE statement as follows:

SELECT DECODE(trl_type, 'I', 'Issue', 'RETN', 'Return', 'RECD', 'Received', 'Unknown') FROM r5translines;

You can also do this using a CASE expression as follows:

SELECT CASE trl_type WHEN 'I' THEN 'Issue' WHEN 'RETN' THEN 'Return' WHEN 'RECD' THEN 'Received' ELSE 'Unknown' END CASE FROM r5translines;

In another selection of data, let's say you want to display the string 'Incoming' for received or returned goods, and the string 'Outgoing' for goods issued. You can use a CASE expression in a slightly different manner as follows:

SELECT CASE WHEN trl_type IN ('RECD', 'RETN') THEN 'Incoming' ELSE 'Outgoing' END CASE FROM r5translines;