Python

Indentation

Python is an indented language, so the code indentation matters. A good practice is to indent with 4 spaces (if you mix spaces and tabs the code won't work.

Operators

Arithmetic

+    Addition
-    Substraction
*    Multiplication
/    Division
**   Power
%    Reminder
//   Floor division (Reminder is removed)

Comparison

==   Equal 
!=   Not equal
>    Greater
>=   Greater or equal
<    Smaller
<=   Smaller or equal
<>   Similar to !=

Assignment

=    Simple assignment
+=   Add and as
-=   Substract and  assignment
*=   Multiply and assignment
/=   Divide and assignment
%=   Modulus and assignment
**=  Exponent and assignment
//=  Floor Divisionn and assignment

Bitwise operators

They perform operations on binary terms. a= 8 → 100; b= 9 → 101; a & b → 100

$    Binary AND
|    Binary OR
^    Binary XOR
~    Binary complement
<<   Binary left shift
>>   Binary right shift

Logic

and
or
not

Membership operators

in
not in

Variables

String

String assingment:
MyString = 'Hello World' OR MyString = "Hello World"
Strings can be subset sliced:

MyString = "Hello world"
MyString[0:4]

Hel

in operator: to check if a substring is contained in a string

String library:

str.lower()
str.upper()
str.capitalize() → Uppercases 1st char
str.center(width[, fillchar])
str.startswith(preffix[, start[, end]])
str.endswith(suffix[, start[, end]])
str.find(substring[, start[, end]])
str.lstrip([chars])
str.rstrip([chars])
str.strip([chars])
str.replace(old, new[, count])see also threading module:

import threading


str.split([char])

len(MyString) → returns the lenght of a string

Collections

List

Collection - allows us to put many values in a single variable. Simple variables are not collections. The previous value is overwritten when changed.

• A List is made up of list 'constants'. Lists are surrounded by square brackets [] and the constants in the list are separated by commas. ([2,4,6,8])
• A List element can be any Python object, even another list
• A List can be empty
• Lists are mutable (they can be changed)
• When len() is used on a list, it counts the number of constants that make up the list. (not the number of characters)
• Lists can be concatenated using +
• Lists can be sliced
• List is a unique type that can be checked using type() (result: <type 'list'>)
• An empty list can be created with list()
• Lists can be tested for contents using in/not in
• List is an ordered sequence
• A list can be sorted with .sort(). Sort changes the list permanently.
• sorted(list) → Returns a list sorted.
• Methods: append, count, extend, index, insert, pop, remove, reverse, sort
• Functions len() - find length, max() - find highest value, min() - find lowest value, sum() - add all values average can be found with sum()/len()

Del Removes an item from a list.
example: a = [1, 2, 3, 4, 5] del a[0] returns [2, 3, 4, 5]

Tuple

Inmutable, declared by ()

• My_tuple = (3, 2, 1)

You can iterate trough tuples.
You can also refer to the items of a tuple like in a list.

My_tuple[0]
3

They support multple assignment, usefull for example to iterate trough dictionaries:

d = dict()
d['steve'] = 2
d['susan'] = 4

for (k, v) in d.items():
    print k, v

Tuples are comparable. They compare the leftmost first, then the next and so on...

>>> (0, 1, 2) < (5, 1, 2)
True
>>> (0, 1, 30000000) < (0, 3, 1)
True

To sort dictionaries by keys.

>>> d = {'c':22, 'b':1, 'a':10}
>>> t = d.items()
>>> t
[('c', 22), ('b',1), ('a', 10)]
>>> t.sort()
>>> t
[('a', 10), ('b',1), ('c', 22)]

To sort by value:

c = {'c':22, 'b':1, 'a':10}
tmp = list()
for k, v in c.items():
    tmp.append( (v, k) )
tmp.sort(reverse = True)
print tmp
[(22, 'c'), (10, 'a'), (1, 'b')]

The same sort by value with list comprehension

c = {'c':22, 'b':1, 'a':10}
print sorted( [ (v,k) for k,v in c.items() ] )

Top 10 most common words.

fhand = open('romeo.txt')
counts = dict()
for line in fhand:
    words = line.split()
    for word in words:
        counts[word] = counts.get(word, 0) +1
lst = list()
for key, val in counts.items():
    lst.append((val, key))
lst.sort(reverse = True)
for val, key in lst[:10]
    print key, val

Dictionary

Key - Value pairs. They are called diferent in diferent languages:
dictionary[key] = value

• Perl → Associative arrays
• Java → Properties, Map or HashMap
• C# → Property bag

Declarationn and assigment:
There are two posible ways to declare them:

• purse = dict()
• puse = {}

purse = dict()
purse['money'] = 12
purse['candy'] = 3

print purse['money']

purse = {'money': 12, 'candy' = 3}

get() method:
If you try to retrieve a value for a key that doesn't exist you would get a traceback error.
To avoid this you should use the get() method, that returns the default value if the key doesn't exist: purse.get(name, default_value) Other methods and functions

• list(purse) → Returns a list of keys.
• dict.keys() → Returns a list of keys.
• dict.values() → Returns a list of values.
• dict.items → Returns a list of tuples ( [(key, value), (key, value)...] )

Word count using files and dictionarys

counts = dict()
file_path = raw_input('Enter file name: ')
file_handle = open(file_path, 'r')
text = file_handle.read()
words = text.split()

for word in words:
    counts[word] = counts.get(word, 0) + 1

bigCount = None
bigWord  = None
for word, count in counts.items():
    if bigCount is None or count > bigCount:
        bigWord = word
        bigCount = count

print 'Most frequent word: ', bigWord, '\nFrequency: ', bigCount'

Conditional

if/elif/else

if a < 10:
    print "Less than 10"
elif a >= 10 and a < 20:
    print "a greater or equal to 10 and less than 20"
else:
    print "a greater or equal to 20"

try/except/finally

try:
    file = open("test.txt")
except:
    print "Could not open file"
finally:
    print "This part will be executed at the end whether the open fails or not"

Loops

For

NumberList = [1, 3, 7, 12, 24]
for number in NumberList:
    print number

Another way:

for i in range(0, len(NumberList)-1):
    print NumberList[i]

while

CtrlNum = 7
while CtrllNum > 3:
    print CtrlNum
    CtrlNum -= 1

Functions

All arguments in Python are passed by reference, if you change a variable value inside a function it will be changed at the calling function.

def MyFunction(formal_arg, optional_arg = None, *variable_lenght_args):
    print formal_arg
    if optional_arg:
        print optional_arg
    for arg in variable_lenght_args:
        print arg

Regular Expressions

   ^     → Matches the beginning of a line
   $     → Matches the end of a line
   .     → Matches any character
  \s     → Matches any whitespace
  \S     → Matches any non-whitespace
   *     → Repeats a character 0 or more times
  *?     → Repeats a character 0 or more times (non-greedy)
  +      → Repeats a character 1 or more times
  +?     → Repeats a character 1 or more times (non-greedy)
 [aeiou] → Matches a single character in the listed set
 [^XYZ]  → Matches a single character NOT in the listed set
[a-z0-9] → The set of characters can include a range
   (     → Indicates where string extraction is to start
   )     → Indicates where string extraction is to end 
  \     → Escape character

Regular Expression Module

It must be imported at the begining of a program:

import re

• re.search(re_string, string) → similar to find()
• re.findall(re_string, string) → similar to find, returns a list
• re.match(re_string, string)

import re
hand = open('mbox-short.txt')
for line in hand:
    line = line.rstrip()
    if re.search('^From:', line);
        print line

Files

Counting lines in a file

fhand = open("words.txt")
count = 0
for line in fhand:
    count = count + 1
print "line count", count

Reading the whole file

fhand =  open("words.txt")
inp = fhand.read() # reads the whole file into memory
print len(inp)# returns the number of characters in a file
print inp # prints the whole file

Searching through a file

fhand= open("mbox-short.txt")
for line in fhand:
    # line = line.rstrip()
    if line.startswith("From:"):
        line = line.rstrip() # better here
        print line

File names and paths

fname = raw_input("Enter a file name: ")
    if fname[0:2] == "~/": #Check to see if it starts with a ~ and a slash
        #If it doesn't start with the ~/, then 
        #the user could be referring to a valid file
        #like "~.py" (I checked: it is possible.)
        #notice below replace is valid on Mac OSX only (and not a good approach overall, cause not portable at all)
        fname = fname.replace('~',"/Users/"+raw_input("Enter your short user name: "),1)
workingfname = fname.replace("\\",'') #This for proper escaping of a valid folder named '~' as '\~', you can also use './~' as Python automatically escapes for you.
#go back to normal program now
handle = open(workingfname,'r') # . . .
for line in handle:
    print line
print "\n"+("That was "+fname+".").center(40)

CSV

import csv

def createCSV(simpleProducts, configProducts, shopName):
    '''
    Remaps a CSV filce created with tiendalistas magento exporter to
    match the Shopify CSV import file
    '''

    counter = 0
    with open(OUTPUT_DIALETCT_TEMPLATE, 'rb') as csvDialectTemplate:
        exportDialect = csv.Sniffer().sniff(csvDialectTemplate.read(1024))

    # Create file and write headers
    with open(OUTPUT_PATH + shopName + '_shopify.csv', 'w') as csvoutFile:
        csvWriter = csv.writer(csvoutFile, dialect=exportDialect)
        csvWriter.writerow(shopyheader)
        counter += 1

        # Write lines for simple products
        for prod in simpleProducts:
            tmp = shopifyCSVitem(prod)
            csvWriter.writerow(tmp.productCSVline())
            counter += 1

            for subP in prod.subProducts:
                    tmp2 = shopifyCSVitem(subP)
                    csvWriter.writerow(tmp2.subProductCSVline())
                    counter += 1

        # Write lines for configurable products
        for prod in configProducts:
            tmp1 = shopifyCSVitem(prod)
            csvWriter.writerow(tmp1.productCSVline())
            counter += 1
            logWrite(logFile, 'Configurable product should be reviewed', tmp1.productCSVline()[0])

            for subP in prod.subProducts:
                tmp2 = shopifyCSVitem(subP)
                csvWriter.writerow(tmp2.productCSVline())
                counter += 1

    return counter

Class

class MyClass:
    __init__(self, arg1, arg2):
        #This functionn will be executed every time you instantiate an element of this class
        self.var1 = arg1
        self.var2 = arg2
    
    def myFunction(self, param1):
        self.var1 = self.var1 +param1

Built in attributes:
__dict__ → Dictionary containing the class's namespace
__doc__ → Class documentation string or None if undefined
__name__ → Class name
__module__ → Module name in which the class is defined. This attribute is "__main__" in interactive mode.
__bases__ → A possibly empty tuple containing the base classes in the order of their occurrence

Inheritance

class SubClassName(ParentClass1[, ParentClass2, ...]):
    'Optional class documentation string'
    # Class code

• issubclass(sub, sup)
• isinstance(obj, Class)

Generic functionality that can be overriden in own classes
__init__(self[, args...]) → Constructor
__del__(self) → Executed when Python's garbage collector destroys an object
__repr__(self) → Evaluable string representation
__str__(self) → Printable string representation
__cmp__(self, x) → Object comparison
__add__(self, other) → To define the + operator behave

Hiding attributes attributes that start with a __ wont be visible to others

os.system()

To execute a Linux command from Python:

os.system(command) → returns exit status

Database Access

MySQLdb

Table creation

#!/usr/bin/python

import MySQLdb

# Open database connection
db = MySQLdb.connect("DB_HOST","DB_USER","DB_PASSWORD","DB_NAME" )

# prepare a cursor object using cursor() method
cursor = db.cursor()

# Drop table if it already exist using execute() method.
cursor.execute("DROP TABLE IF EXISTS EMPLOYEE")

# Create table as per requirement
sql = """CREATE TABLE EMPLOYEE (
         FIRST_NAME  CHAR(20) NOT NULL,
         LAST_NAME  CHAR(20),
         AGE INT,  
         SEX CHAR(1),
         INCOME FLOAT )"""

cursor.execute(sql)

INSERT operation

#!/usr/bin/python

import MySQLdb

# Open database connection
db = MySQLdb.connect("DB_HOST","DB_USER","DB_PASSWORD","DB_NAME" )

# prepare a cursor object using cursor() method
cursor = db.cursor()

# Prepare SQL query to INSERT a record into the database.
sql = "INSERT INTO EMPLOYEE(FIRST_NAME, \
       LAST_NAME, AGE, SEX, INCOME) \
       VALUES ('%s', '%s', '%d', '%c', '%d' )" % \
       ('Mac', 'Mohan', 20, 'M', 2000)
try:
   # Execute the SQL command
   cursor.execute(sql)
   # Commit your changes in the database
   db.commit()
except:
   # Rollback in case there is any error
   db.rollback()

# disconnect from server
db.close()

Read example Once the query is made you can:

• fetchone()
• fetchall()
• rowcount

#!/usr/bin/python

import MySQLdb

# Open database connection
db = MySQLdb.connect("localhost","testuser","test123","TESTDB" )

# prepare a cursor object using cursor() method
cursor = db.cursor()

# Prepare SQL query to INSERT a record into the database.
sql = "SELECT * FROM EMPLOYEE \
       WHERE INCOME > '%d'" % (1000)
try:
   # Execute the SQL command
   cursor.execute(sql)
   # Fetch all the rows in a list of lists.
   results = cursor.fetchall()
   print "Found ', cursor.rowcount, ' results: ' 
   for row in results:
      fname = row[0]
      lname = row[1]
      age = row[2]
      sex = row[3]
      income = row[4]
      # Now print fetched result
      print "fname=%s,lname=%s,age=%d,sex=%s,income=%d" % \
             (fname, lname, age, sex, income )
except:
   print "Error: unable to fecth data"

# disconnect from server
db.close()

PostgreSQL

Refer to: PostgreSQL

Other

Refer to: Database Interfaces

ssh tunneling

import subprocess
sshcommand = ['ssh', '-o', 'StrictHostKeyChecking=no', '-i', keyFilePath, '-N', '-L', '8888:localhost:3306', 'user@' + Host]
ssh_proc = subprocess.Popen(sshcommand, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
# Do stuff
ssh_proc.terminate()

Multithreading

#!/usr/bin/python

import thread
import time

#Define a function for the thread
def print_time( threadName, delay):
   count = 0
   while count < 5:
      time.sleep(delay)
      count += 1
      print "%s: %s" % ( threadName, time.ctime(time.time()) )

# Create two threads as follows
try:
   thread.start_new_thread( print_time, ("Thread-1", 2, ) )
   thread.start_new_thread( print_time, ("Thread-2", 4, ) )
except:
   print "Error: unable to start thread"

while 1:
   pass

(*) see also threading module:

import threading