Category Archives: Scripts
Linux performance monitoring (an introduction)

1. Classification

Before getting into action, let’s split the “performance” problem in a couple of boxes, as the concept itself is quite general. First, deciding what we want to monitor (an entire system? a particular application?) – and second, deciding on what type of performance monitoring do we require (stats collection by the kernel? in-depth analysis?). Based on this particular classification, we may end up with 4 categories, each with its particular software selection:

 

Stats (Counters)

Tracing / Profiling / Debugging

 System Wide

 Per Process

NB:

  • netstat offers much more info beyond statistics on interface / protocol and may also be used to monitor individual connections.

  • dtrace and SystemTap can also trace individual applications.

Continue Reading →

Python Cheat Sheet: Lambdas

Did you think that list comprehensions were a complicated thing? I suspect you’ll think again about complicated things in Python while reading this text.

Lambdas

By definition, lambdas are short functions defined in place, used for small data processing tasks. There are 2 reasons on why they should be used:

  • Execution speed – they can be optimized by compilers, first by removing an actual function call, next by opening the door for more optimizations through any possible internal (by the compiler) code re-arrangement;

  • Writing less code.

A Python example of a n square lambda:

g = lambda x: x ** 2	# e.g. g(3) yields a value of 9

Lambdas are usually used in conjunction with data processing functions such as filter, map and reduce.

Filtering

If we want to select only a portion of some input, the filter function comes to its best use in combination with a lambda:

print filter(lambda x: x % 2 == 0, xrange(0, 11))
...
[0, 2, 4, 6, 8, 10]
Transformation

Applying a transformation function to the entire input is a job for map:

print map(lambda x: x * 2, xrange(0, 11))
...
[0, 2, 4, 6, 8, 10, 12, 14, 16, 18, 20]
Summarizing

Processing the entire input in order to get a single final result is a job for reduce. Please note that 2 arguments are required for the lambda function used for processing; the first 2 elements are used in the beginning, then the lambda result and the next element are used until the input is exhausted.

print reduce(lambda x, y: x + y, xrange(0, 11))
...
55	# 1 + 2 + ... + 10

Now for some serious stuff:

Continue Reading →

Python Cheat Sheet: Dictionaries

Dictionaries are Python implementations of hash tables. They store arbitrary key-value associations, not restricted to a single data type. The dictionaries do not preserve order (no order between elements is defined) so there is no guarantee that any element enumeration will yield a certain order.

NB: unless otherwise specified, everything is Python 2 syntax.

Defining a dictionary:
D = { }
Initializing a dictionary:
D = dict( (i, False) for i in xrange(1, 11) if i % 2 == 0 )	#Python 2
D = { i : False for i in range(1, 11) if i % 2 == 0 }		#Python 3

Note: this syntax is also known as dictionary comprehension.

Enumerating all the items in a dictionary:
for i in D:
	print i, D[i]	#key, value

…or:

for k, v in D.iteritems():
	print k, v	#key, value
Looking up an element:
if k in D:
	print k 
Adding elements:
D[key] = value

Note: no error is thrown if the element already exists; an overwrite happens.

Removing elements:
del D[key]
Using a dictionary as a lookup table to remove duplicates in a list:
D = {}
for i in L:
	if i in D:
  		del L[i]
  	else:
  		D[i] = True

Note: a set data type is more suited for such task. There is also a simplified syntax (e.g. list(set(L)) ) that can be used if the element order within the input list does not need to be preserved.

That’s it for today, thank you for your read!

Previous Page · Next Page