(Python) PyProgram #8 - Cycling Alphabets

Kushal came down with a simple looking problem with constraints that might make solving it a bit not-a-needle-into-the-banana. As usual, me and Jacob sat down trying to solve it. Rather try finding various ways of solving it. The problem is, given one character within the range a-z or A-Z (not as a list, but from string.letters) we have to find the next character in the alphabetical order. When ‘z’ or ‘Z’ is encountered, the next alphabet returned should be ‘a’ or ‘A’ respectively. The constraint is that if or while conditions and for loops should not be used. And more importantly, the main solution should be of a single line. Though Kushal had a more mathematical solution, this is what me and Jacob managed to cook up as a more logical solution (note the ‘and’ and ‘or’ ),

import string

def thenext(theletter):
    return ((string.letters.find(theletter)+1) %26) and (string.letters[string.letters.find(theletter)\
+1]) or (string.letters[string.letters.find(theletter)-25])

print "for 'a' i get: "+thenext('a')
print "for 'g' i get: "+thenext('g')
print "for 'z' i get: "+thenext('z')
print "for 'A' i get: "+thenext('A')
print "for 'H' i get: "+thenext('H')
print "for 'Z' i get: "+thenext('Z')

The solution looks like..

for 'a' i get: b
for 'g' i get: h
for 'z' i get: a
for 'A' i get: B
for 'H' i get: I
for 'Z' i get: A

the new KDE era begins

All Ks are excited and rather celebrating. Wonder why? Because their new baby is out and it’s rocking the world. Still have no clue of what am excited about too!? It’s all about the new version of KDE , the KDE 4.0 release which happened yesterday. This has been the work of lots and lots of contributors to make a yet-another-endeavor to bless this world with a rocking Desktop Environment. As KDE has always been doing, this time too it has come out with really awesome desktop. The additional good news is that the KDE 4.0 release announcement is also available in various languages like Hindi, Bengali (India), Malayalam, Gujarati, Marathi and Tamil (coming out soon!).

It’s time for the party, one is indeed happening at Mumbai and one is being planned in Chennai as well. Thanks and wishes to all the Ks who have worked to bring out this wonderful version of KDE. Thanks also to those wonderful buddies of mine who translated the release announcement in various languages and to the one who pitched in this whole idea of having localized version of the release announcement

KDE 4.0, its roKKKKKing!

(Python) PyProgram #7 - Time Object Diffs

I had two time objects and needed to find the difference of time between them. Upon Googling, querying in #python and checking the documentation, found that there is not built-in function available for this. Though there were some cookbook recipes, I wanted to write my own solution to find the difference between two datetime.time objects. Though datetime.timedelta was for time difference requirements it only has a days, seconds and microseconds property while I need hour, minute and seconds. Thus, I wrote my timediff method which accepts two time objects btime and stime where btime is a bigger time than stime (b for big and s for small ), and it returns a datetime.time object for the difference time. (Why I need to have time objects? I use storm ORM )

def timediff(btime, stime):

    """Difference between two datetime.time objects

    Accepts two datetime.time objects where btime > stime

    Returns a datetime.time object of the difference in time of

    the two datetime objects.

    """

    btdelta = timedelta(hours=btime.hour, minutes=btime.minute, seconds=btime.second)

    stdelta = timedelta(hours=stime.hour, minutes=stime.minute, seconds=stime.second)

    tdiff = btdelta - stdelta

    tdiffsec = tdiff.seconds

    if tdiffsec < 60 and tdiffsec > 0:

        return time(0, 0, int(tdiffsec))

    elif tdiffsec < 3600 and tdiffsec > 0:

        tdiffsplit = str(tdiffsec/60.0).split('.')

        tdiffmin = int(tdiffsplit[0])

        tdiffsec = float("0."+tdiffsplit[1])*60

        return time(0, int(tdiffmin), int(tdiffsec))

    elif tdiffsec > 0:

        tdiffhourmin = str(tdiffsec/3600.0).split('.')

        tdiffhour = int(tdiffhourmin[0])

        tdiffminsec = str(float("0."+tdiffhourmin[1])*60).split('.')

        tdiffmin = int(tdiffminsec[0])

        tdiffsec = float("0."+tdiffminsec[1])*60

        return time(tdiffhour, tdiffmin, int(tdiffsec))

    else:

        return time(0, 0, 0)

[Bash] Quick file renaming

I had a bunch of .deb packages from the apt archive. But the problem was that in the names of the packages, there was a “%3″ in the places where there should be a “_”. We found that the symbol was corresponding to the underscore character and it was somehow getting replaced in the name. Hence we needed to rename all filenames, changing the ‘%3′ to ‘_’ and I was requested to write a quick script. As my usual bash buddy was busy, I resolved to some Google’ing and found this.

I first started by trying to echo the file name, then replace the ‘%3′ with ‘_’, print it, and finally rename the file.

for FILE in `ls`; do echo $FILE; done;
for FILE in `ls`; do NEWFILE=`echo $FILE | sed 's/%3a/_3a/g'`; echo $NEWFILE; done;
for FILE in `ls`; do NEWFILE=`echo $FILE | sed 's/%3a/_3a/g'`; mv "$FILE"  $NEWFILE ; done;

But when I execute my final script, it informs that the oldfile and newfile to the `mv` command are the same. Think the `sed` command itself renamed the file. Anyways, it did the job I wanted it to

Updates from floyd_n_milan:

for file in *; do mv -v “$i” “${i/\%3a/_3a/}”; done;


(Python) PyProgram #6 - Finding weeks of a month

My colleague had a problem, he needed to separate the dates in a month according to weeks. That is, given a month, he has to find the dates in 1st week, dates in 2nd week and so on. This is need to query and collect a weeks data for a given month. He was trying to write his own solution which we Python guys around thought was a bit non-straight forward. My other colleague Jacob sat and wrote a small Py script to order the dates per week of a given month,

from datetime import datetime, timedelta

current_date = datetime.now() year, month = current_date.timetuple()[:2]

first_day_of_month = datetime(year, month, 1)

if month == 12:

    first_day_of_next_month = datetime(year+1, 1, 1)

else:

    first_day_of_next_month = datetime(year, month+1, 1)

#Subtracting 1 day

last_day_of_month = first_day_of_next_month - timedelta(1)

#First and Last days of the month

print first_day_of_month, last_day_of_month

#Computing start and end week days

first_week_day = first_day_of_month.weekday()+1

date_range = [0]*first_week_day + range(first_day_of_month.day,

last_day_of_month.day+1)

month = []

while date_range:

    if len(date_range) >= 7:

        week = date_range[:7]

        date_range = date_range[7:]

    else:

        week = date_range

        date_range = None

     month.append(week)

for week in month:

    print week

for week in month:

    start_week, end_week = min(week), max(week)

    if start_week==0:

        start_week = 1

    print start_week, end_week

Python BoF at foss.in

I was enquiring Baiju about the Python Collective at the Bar Camp Bangalore 5, which is scheduled for this weekend, as I hadn’t seen any discussion about it in the BangPypers list. This is when Baiju asked me about having a BoF during foss.in, which I readily seconded and as per his advice, start the discussion in BangPypers list for the same. And thats why this post comes out.

We plan to have a “Python BoF” during foss.in 2007, organized by BangPypers (the Bangalore Python User Group). The discussions will happen in the BangPypers Mailing List to decide the day and time, after which I will update the tentative date and time here, as well announce it in both BangPypers and foss.in mailing lists. Hope to see a lot of Pythonians at the BoF and interesting discussions happen

(Python) ipython

When I was just peeping into the discussions at #python, I saw a few talking about ipython. On first look, I thought it to be Iron Python stuff but was wrong. It is an (another?) enhanced Python Shell or what we otherwise call as Interactive Python Mode. The good thing with these stuffs is you can just download and try them, not much a paining process to get them working. Hence I went for it.

Features

• Dynamic Object Introspection : We can access docstrings, function definition prototypes, source codes of modules, source files and a lot more dynamically.

For example, after importing `os` module, typing os.path?? or os.path? gives complete source code and description of the os module respectively. Similarly `%pdoc object`gives us the object’s doctstring information.

• Local namespace completion with TAB press.
• Numbered input/output prompts.

In [1]: print “Hello World!”
Hello World!

In [2]: (1+2)*3
Out[2]: 9

• User-extensible`magic` commands (type %magic for information).
• System shell access with ! prefix.

!ls will issue the `ls` command in the system shell and print its output.

• File system navigation with %cd command.
• Verbose and colored exception traceback.
• Auto-parenthesis.

>>> callable_ob arg1, arg2, arg3
and the input will be translated to this:
–> callable_ob(arg1, arg2, arg3)

• Auto-quoting.

>>> ,my_function a b c # becomes my_function("a","b","c")
>>> ;my_function a b c # becomes my_function(”a b c”)

• Embeddable within other python code/modules.

Installation

I always prefer the `svn checkout` way of getting the source code. We can get to know more about ipython at its home site. We can also get pythonreadline from the same site, which helps us to have features like color, tab, completion, etc. (I read somewhere that it is not needed for ipython as it implicitly has pyreadline). The following are the steps to install ipython from the svn repos as a local copy,

1. We can get the source code by doing an svn checkout,

svn co http://ipython.scipy.org/svn/ipython/ipython/trunk ipython

2. We need to get into the checked out source directory and run the following command

python setup.py install

3. The –home option can be used to specify a directory to install the ipython, else it will install in the default python directories.

python setup.py install --home /home/myuser/python/ipython

4. If we are using –home option, then we need to add the following line into our.bashrc. Else we need to export the PYTHONPATH variable every time adding the path of our ipython/lib/python installation to it.

export PYTHONPATH=/home/myuser/python/ipython/lib/python

5. We can run ipython by executing ipython/bin/ipython from the shell (or create a link pointing to this file).

The complete documentation can be obtained as pdf as well as viewed as HTML.

(Bash) My Top 10 Commands - September 2007

Long since I looked at what commands I frequently use, I decided to re-run that small command I got to learn some months ago. This shows am working a lot on files and that too on some other machine through networking

$ history | awk ‘{print $2}’ | awk ‘BEGIN {FS=”|”}{print $1}’ | sort | uniq -c | sort -n | tail | sort -nr

142 ls
64 cd
60 vim
36 exit
25 sudo
24 ssh
15 scp
14 find
13 svn
13 apt-cache

(Bash) Copy with Find

Wanted to copy all my EPM .list files to a separate folder so I can do a bit of edit and test with it. Wanted to do it in a true geek mode and hence decided to make use of bash’s find command (Thanks to floyd_n_milan for introducing me to this wonderful command). Since it some time I really used `find`, I had to do a bit of Googling to find I needed. This might be a piece of piss for seasoned Bash’ers but for me it was a chance to revive my detoriating Bash knowledge and as it always happens, I love to post newbie stuffs.

$ find ./myproj/pkgs -name \*.list -exec cp {} epm-list/ \;

This command searches the myproj/pkgs directory (within current directory) and its sub-directories for files ending with .list. Upon each file found, it executes the `cp` command copying the file to the directory epm-list. The {} is replaced by the name of the file found (each time).

Please note the escaping ‘\’ before the ‘*’ in the regular expression and before the trailing ‘;’. If you miss the trailing semicolon or the escaping, bash will complain that the -exec has no options. We need not put `epm-list/{}`, it uses the file name automatically (if we use {} then it tries to put the file in epm-list/<path-where-the-file-is-actually-found> which ends in a path not found error).

(Python) Function calls for beginners

A function `call` is a callable object with a series of arguments which can be empty.

syntax : primary(arg_list['',''])
arg_list ::= positional_args [, keyword_ags][, *expression][, ** expression] |
keyword_args[, *expression][, ** expression] |
*expression [, **expression] |
** expression

positional_args ::= expression (, expression*)
keywords_args ::= keyword_item (, keyword_item*)
keyword_item ::= identifier = expression

`primary` is the callable object (function name, built-in callable method objs, class objs, methods of class instances and callable class instances)

How arguments are processed ?

1. a list of unfilled slots for formal parameters is created
2. N positional_args are placed in first N slots
3. keyword_args are placed in corrsponding slot using identifier

TypeError if a slot is already filled

4. Unfilled slots are filled with corresponding default values

TypeError if slots exist with no default values
TypeError if positional_args > formal param slots
TypeError if ketyword_args doesn’t match with formal param name unless **identifier is present

5. *expression evaluates to sequence like y1…yM and are placed as additional positional_args after x1…xN such that the call is made with x1….xN,y1..yM positional arguments
6. **expression is a dictionary contained excess keyword arguments

TypeError if **expression doesn’t evaluate to dictionary

7. *expression is processed before keyword arguments, hence unusual to use both keyword_args and *expression

A call return can be,

• as per the return statement for a user-defined func

• up to the interpreter for a build-in func or method

• a new class instance for a class object

• the user-defined function for a class instance method with the instance as the first argument

• as per __call__() method for a class instance

Examples:

# define a function with two params, printing them as result
>>> def f(a,b): print a, b
...

# both are keyword_args
>>> f(a=1,b=2)
1 2

# both are positional_args
>>> f(1,2)
1 2

# a is keyword and b is considered positional. ERROR!
>>> f(a=1,2)
File "<stdin>", line 1
SyntaxError: non-keyword arg after keyword arg

# a is positional and b is keyword_arg
>>> f(1,b=2)
1 2

# 1 is positional while 2 is *expr and takes as additional positional_arg for b
>>> f(1,*(2,))
1 2

# 1 is keyword for a, but 2 is also positional_arg for a. ERROR!
>>> f(a=1,*(2,))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: f() got multiple values for keyword argument 'a'

# 1 is a keyword_arg for b, while 2 is positional_arg for a
>>> f(b=1,*(2,))
2 1

# 1 is keyword_arg for a, while 2 is specified thru **expr for b
>>> f(a=1,**{'b':2,})
1 2

# 1 is positional_arg using *expr for a, b is keyword_arg thru **expr for b
>>> f(*(1,),**{'b':2,})
1 2

# both a and b are specified as keyword_args thru **expression
>>> f(**{'a':1,'b':2,})
1 2

Original Reference (Python Reference Documentation)

(Bash) My Top 10 Commands - June 2007

Following the posts in Planet Ubuntu, I thought of trying out the same to find out which commands I have often used recently. So I tried..

$ history | awk ‘{print $2}’ | awk ‘BEGIN {FS=”|”}{print $1}’ | sort | uniq -c | sort -n | tail | sort -nr

And the result was..

81 ls
68 sudo
64 cd
33 apt-cache
26 exit
24 screen
22 vim
12 ping
10 cat
8 svn

Who Uses Python ?

I happen to see an interesting post in the Python mailing list. The question was to know who uses Python, other than sys-admins and web developers. And here is a consolidation of the replies till now.

• For login scripts, locking down PCs, automating backups, GIS, and more.
• Electrical engineering. It’s pretty handy for writing programs to talk to embedded systems using various protocols/interface (async-serial, ethernet, etc.). It’s also good for analyzing communications, analyzing performance tests for analog instruments, and so on.
• The Technical Artists are using Python more and more for development needs.
• For digital art (music visualization). It’s excellent for data analysis (I’ve done everything from stats on lines in an SQL database to mining flat text files of data for statistical projections of MLB baseball player performance). I know a couple of
  people who sell a double-entry accounting system written in Python, which is presumably “finance”. Web analytics is very common (I’ve seen several such projects)
• we use Python for controlling fully automated logistics solutions (conveyors and stacker cranes), for generating PLC code etc.

There is also a very long reply on how Python has been used for Railway Computer-control Simulation which lists the use of Python 2.5 as,

• The main database where all states are kept (signal aspects, given and actual turnout states, track occupation …). It’s a simple binary protocol using TCP; the server maintains string:int pairs in a Python dict. It’s made with Twisted and running under twistd.

• The automatic drive controlling software of the model trains. It listens for changes of signals and track occupation and controls the model trains using a commercial digital model railroad controller attached to /dev/ttyS0. Also using Twisted/twistd, and
  pyserial.

• Various helper scripts and little servers for small functional units (tramway reverser, level crossings)

This is the latest one I read,

• To manage the database application, data integration, and reporting
• to shuffle and sort important files around, to convert reports into formats necessary for various import and export functions, to reconcile data between database systems, and to maintain little text databases for various business processes.

Some other common replies includes text search, data mining and biotechnology. Nice to hear about some real time uses of Python, different from the usual administration and web stuffs.

Positional Parameters in Bash

Been long since I posted something on Bash and as usual, I again post something from Mrugesh’s (floyd_n_milan) key board. This time its an explanation of Positional parameters in Bash, which was actually a reply to a question in GLUG-BOM mailing list. I felt the explanation to be more clear than what I found in many books on the same, hence here goes his explanation (slightly edited).

$* expands the positional parameters. 

Positional parameters are arguments passed to a command.
When you run your script by doing, say, 

$ ./image-resize image1 image2 image3 

the positional parameters get set to ./image-resize,
image1, image2 and image3. 

The positional parameters can be used as the variables
$0 to $n where n is any integer. $0 is always set to
the name of the script, in this case image-resize.
The parameters from $1 onwards are the actual
arguments. So in this case, 

$1 == image1
$2 == image2 and
$3 == image3 

These are read only variables. You can't manually
assign a value to them. You must use either set or shift. 

When you process the arguments, you can either call them
individually, or, as is most common, process them in a
loop, one by one. You've used the second method. 

$* and $@ are two variables that can be used to query
all the positional parameters together. When unquoted,
they are subject to IFS word splitting like any other
variable in bash. What this means is that if your arguments
have spaces or newlines embedded in them, they'll be
split up to be two different entities. 

The safe way is to double quote ( " ) the variables,
which prevents this internal splitting. 

Mind the double quotes. VERY important. 

Here's an example: 

$ set "foo bar" baz blah
$ # "foo bar" is intended to be a single argument.
$ # set can be used to set various bash options and
   positional parameters.
$ echo $1
foo bar
$ echo $2
baz
$ echo $3
blah
$ for i in $@; do echo '==> '${i}' <=='; done
==> foo <==
==> bar <==
==> baz <==
==> blah <==
$ # Notice that foo and bar have been split up. Undesirable.
$ for i in $*; do echo '==> '${i}' <=='; done
==> foo <==
==> bar <==
==> baz <==
==> blah <==
$ # When unquoted $@ and $* produce the same result.
$ for i in "$@"; do echo '==> '${i}' <=='; done
==> foo bar <==
==> baz <==
==> blah <==
$ # I suppose this is what you wanted?
$ for i in "$*"; do echo '==> '${i}' <=='; done
==> foo bar baz blah <==
$ # When double quoted, $@ and $* differ vastly..
$ # And that's not the result you wanted either.
$ # Always use "$@".

And here is a small addition from cydork,

<cydork> add that when you use “$*” the positional parameters will be separated with first char of IFS

And one additional point from Mrugesh himself,

the ${array[@]} and ${array[*]} behave in a similar manner

PyProgram #5 - SMTP Mail Sender

Had some requirement to send mails in my project, so we were digging into the smtplib module. Thought of trying out a small script which will send mails. The example code was fine, but modified the functions and added “Subject: ” header to it. Tested it and voila it sent mails A rather trivial piece of code, but posting it to save it somewhere in my web space

#!/usr/bin/env python

"""Command-line SMTP Mail Sender
This script can send mails provided you have an account
with an SMTP server.

The inputs have to be given at the
command line. The recepients address can be a space
separated list of emails.

The subject can be multiple lines, seperated by empty lines.
When done, end the subject with Ctrl+D to send the mail.
"""

import smtplib

# function to fetch the input from command line
def prompt(prompt):
return raw_input(prompt).strip()

fromaddr = prompt("From: ")
toaddrs = prompt("To: ").split()
subject = prompt("Subject: ")
print "Enter message, end with ^D (Unix):"

# Add the From: and To: headers at the start
msg = ("From: %snTo: %snSubject: %snn" %(fromaddr, ", ".join(toaddrs), subject))
while 1:
try:
line = raw_input()
except EOFError:
break
if not line:
break
msg = msg + line

print "Message length is " +repr(len(msg))

# connect and send the mail
server = smtplib.SMTP()
# connects to default port, if not specify it as additional argument
server.connect('mail.myhost.com')
# specify username and password
server.login('me.myhost.com','me1234me')
server.set_debuglevel(1)
server.sendmail(fromaddr, toaddrs, msg)
server.quit()
# done!

PyProgram #4 - Who is on duty ?

We had an interesting problem posed in python mailing list, though it is not a very tough one. But still, practical problems are a fun to solve. This guy wanted to know how he can find which group is on duty. There are 4 group of fire-fighters each cycled on duty for a day one after another. Poor guys, they do not get any off even for holidays. The 4 groups all joined on a single day, so the start day is common for all. Based on this starting day, we have to find which group is on duty if given a particular day. This is a simple and interesting problem to solve. So, we all jumped in and used the datetime module. Here is my code dutycycle.py,

#!/usr/bin/env python
"""Data Cycle Finder
Find which group takes charge on a corresponding date.
There are four groups, each working on a 24-hour cycle
one after another. The groups work one day after another
in order, without any break for holidays.

Given a date, it is needed to find which group is on
duty. All the groups have a start date and a group
number. Hence given a date, the output should be the
group number which is on duty.

The input date is of the form dd/mm/yyyy.
"""

from datetime import date
import sys

def dutyfinder(_date):
    start = date(2007,1,1) # the duty cycle started on 1st jan 2007
    return (date.toordinal(_date)-date.toordinal(start))%4

inputdate = sys.argv[1]
indate = sys.argv[1].split('/')
enddate = date(int(indate[2]),int(indate[1]),int(indate[0]))
onduty = int(dutyfinder(enddate)) + 1
print "The group to report for duty on %s is %s" %(str(inputdate),str(onduty))

Now, we need to call this file with the input date as the argument. The starting date is set to 1st January 2007 and the input date format is dd/mm/yyyy. To find which group should report to duty on 10.01.2007,

$ python datacycle.py 10/01/2007

The group to report for duty on 10/01/2007 is 2

I had mailed the OP the code and he has thanked me for it