1. The default initializer gotcha
Suppose you are in an interview setting and you are being handed a piece of paper with the following Python code written on it:
def foo(bar=[])
No explanations, no nothing, just the obvious question: what is wrong with the code above?
… nothing in particular. Yes, nothing is wrong with the code above, it is legal Python code (apart from missing the ending “:” and that no function body is given). Who am I to judge the particular use case – after all, a bug is defined as code that does not run the way the programmer expected it to.
But suppose we add more code to that function definition:
def foo(bar=[]): bar.append(1) return bar
Now things are getting messier. What would a “normal” Python programmer expect to be the result of calling foo() with no arguments for 3 times in a row?
foo() # [1] foo() # [1] foo() # [1]
Nope. The real result is more similar to:
foo() # [1] foo() # [1, 1] foo() # [1, 1, 1]
How is this even possible? The issue is with how default initializers for function arguments are handled in Python and this is completely different from C/C++. The scope of the default initializer is not the function body but rather the global scope (something similar to the static variables in C/C++). And static variables are good for caching and keeping state. But yes, I believe we had enough of this.
1. Classification
Before getting into action, let’s split the “performance” problem in a couple of boxes, as the concept itself is quite general. First, deciding what we want to monitor (an entire system? a particular application?) – and second, deciding on what type of performance monitoring do we require (stats collection by the kernel? in-depth analysis?). Based on this particular classification, we may end up with 4 categories, each with its particular software selection:
Stats (Counters) | Tracing / Profiling / Debugging | |
System Wide | ||
Per Process |
NB:
netstat offers much more info beyond statistics on interface / protocol and may also be used to monitor individual connections.
dtrace and SystemTap can also trace individual applications.
Did you think that list comprehensions were a complicated thing? I suspect you’ll think again about complicated things in Python while reading this text.
Lambdas
By definition, lambdas are short functions defined in place, used for small data processing tasks. There are 2 reasons on why they should be used:
Execution speed – they can be optimized by compilers, first by removing an actual function call, next by opening the door for more optimizations through any possible internal (by the compiler) code re-arrangement;
Writing less code.
A Python example of a n square lambda:
g = lambda x: x ** 2 # e.g. g(3) yields a value of 9
Lambdas are usually used in conjunction with data processing functions such as filter, map and reduce.
Filtering
If we want to select only a portion of some input, the filter function comes to its best use in combination with a lambda:
print filter(lambda x: x % 2 == 0, xrange(0, 11)) ... [0, 2, 4, 6, 8, 10]
Transformation
Applying a transformation function to the entire input is a job for map:
print map(lambda x: x * 2, xrange(0, 11)) ... [0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20]
Summarizing
Processing the entire input in order to get a single final result is a job for reduce. Please note that 2 arguments are required for the lambda function used for processing; the first 2 elements are used in the beginning, then the lambda result and the next element are used until the input is exhausted.
print reduce(lambda x, y: x + y, xrange(0, 11)) ... 55 # 1 + 2 + ... + 10
Now for some serious stuff: