Difference between revisions of "Python"

From RHS Wiki
Jump to navigation Jump to search
Tag: visualeditor
 
(59 intermediate revisions by the same user not shown)
Line 1: Line 1:
== Indentation ==
+
====[[Python: Install|Install]]====
 +
 
 +
==Indentation==
 
Python is an indented language, so the code indentation matters. A good practice is to indent with 4 spaces (if you mix spaces and tabs the code won't work.
 
Python is an indented language, so the code indentation matters. A good practice is to indent with 4 spaces (if you mix spaces and tabs the code won't work.
== Operators ==
+
==Operators==
=== Arithmetic ===
+
===Arithmetic===
 
  <nowiki>+    Addition
 
  <nowiki>+    Addition
-    Substraction
+
          -    Substraction
*    Multiplication
+
          *    Multiplication
/    Division
+
          /    Division
**  Power
+
          **  Power
%    Reminder
+
          %    Reminder
//  Floor division (Reminder is removed)</nowiki>
+
          //  Floor division (Reminder is removed)</nowiki>
=== Comparison ===
+
===Comparison===
 
  <nowiki>
 
  <nowiki>
==  Equal  
+
          ==  Equal  
!=  Not equal
+
          !=  Not equal
>    Greater
+
          >    Greater
>=  Greater or equal
+
          >=  Greater or equal
<    Smaller
+
          <    Smaller
<=  Smaller or equal
+
          <=  Smaller or equal
<>  Similar to !=</nowiki>
+
          <>  Similar to !=</nowiki>
=== Assignment ===
+
===Assignment===
 
  <nowiki>
 
  <nowiki>
=    Simple assignment
+
          =    Simple assignment
+=  Add and as
+
          +=  Add and as
-=  Substract and  assignment
+
          -=  Substract and  assignment
*=  Multiply and assignment
+
          *=  Multiply and assignment
/=  Divide and assignment
+
          /=  Divide and assignment
%=  Modulus and assignment
+
          %=  Modulus and assignment
**=  Exponent and assignment
+
          **=  Exponent and assignment
//=  Floor Divisionn and assignment</nowiki>
+
          //=  Floor Divisionn and assignment</nowiki>
  
=== Bitwise operators ===
+
===Bitwise operators===
 
They perform operations on binary terms. a= 8 → 100; b= 9 → 101; a & b → 100
 
They perform operations on binary terms. a= 8 → 100; b= 9 → 101; a & b → 100
 
  <nowiki>
 
  <nowiki>
$    Binary AND
+
          $    Binary AND
|    Binary OR
+
          |    Binary OR
^    Binary XOR
+
          ^    Binary XOR
~    Binary complement
+
          ~    Binary complement
<<  Binary left shift
+
          <<  Binary left shift
>>  Binary right shift</nowiki>
+
          >>  Binary right shift</nowiki>
  
=== Logic ===
+
===Logic===
 
  <nowiki>
 
  <nowiki>
and
+
          and
or
+
          or
not</nowiki>
+
          not</nowiki>
  
=== Membership operators ===
+
===Membership operators===
 
in<br />
 
in<br />
 
not in<br />
 
not in<br />
  
== Variables ==
+
==Variables==
=== Numbers ===
+
===Numbers===
=== String ===
+
 
 +
====0 padding 2 digits====
 +
<syntaxhighlight lang="python3">
 +
print("{:02d}".format(1))
 +
</syntaxhighlight>
 +
 
 +
===String===
 
String assingment:<br />
 
String assingment:<br />
 
MyString = 'Hello World'  OR  MyString = "Hello World"<br />
 
MyString = 'Hello World'  OR  MyString = "Hello World"<br />
Line 75: Line 83:
 
str.rstrip([chars])
 
str.rstrip([chars])
 
str.strip([chars])
 
str.strip([chars])
str.replace(old, new[, count])
+
str.replace(old, new[, count])see also threading module:
 +
 
 +
import threading
 +
 
 +
 
 +
str.split([char])
 
</source>
 
</source>
  
 
len(MyString) → returns the lenght of a string
 
len(MyString) → returns the lenght of a string
  
=== Collections ===
+
===Collections===
==== List ====
+
====List====
 +
Collection - allows us to put many values in a single variable. 
 +
Simple variables are not collections. The previous value is overwritten when changed. 
  
* sorted(list) → Returns a list sorted.
+
*A List is made up of list 'constants'. Lists are surrounded by square brackets [] and the constants in the list are separated by commas. ([2,4,6,8])
 +
*A List element can be any Python object, even another list
 +
*A List can be empty
 +
*Lists are mutable (they can be changed)
 +
*When len() is used on a list, it counts the number of constants that make up the list. (not the number of characters)
 +
*Lists can be concatenated using +
 +
*Lists can be sliced
 +
*List is a unique type that can be checked using type() (result: <type 'list'>)
 +
*An empty list can be created with list()
 +
*Lists can be tested for contents using in/not in
 +
*List is an ordered sequence
 +
*A list can be sorted with .sort(). Sort changes the list permanently.
 +
*sorted(list) → Returns a list sorted.
 +
*Methods: append, count, extend, index, insert, pop, remove, reverse, sort
 +
*Functions len() - find length, max() - find highest value, min() - find lowest value, sum() - add all values average can be found with sum()/len()
 +
*reversed list:
 +
**list.reverse()
 +
**list[::-1]
 +
**reversed(list)
  
==== Tuple ====
+
'''Del'''
 +
Removes an item from a list.<br />
 +
example: a = [1, 2, 3, 4, 5] del a[0] returns [2, 3, 4, 5]
 +
 
 +
<br />
 +
 
 +
====List comprehension====
 +
flatten list<syntaxhighlight lang="python3">
 +
def flatten(list_of_lists):
 +
    return [element for secondary_list in list_of_lists for element in secondary_list ]
 +
 
 +
a = [[1,2,3], [3,4,5], ["a", "b"]]
 +
flatten(a)
 +
 
 +
Out[29]: [1, 2, 3, 3, 4, 5, 'a', 'b']
 +
</syntaxhighlight><br />
 +
 
 +
====Tuple====
 
Inmutable, declared by ()
 
Inmutable, declared by ()
* My_tuple = (3, 2, 1)
+
 
 +
*My_tuple = (3, 2, 1)
 +
 
 
You can iterate trough tuples.<br />
 
You can iterate trough tuples.<br />
 
You can also refer to the items of a tuple like in a list.<br />
 
You can also refer to the items of a tuple like in a list.<br />
Line 109: Line 161:
 
True
 
True
 
</source>
 
</source>
This can be used to sort dictionaries by keys.
+
'''To sort dictionaries by keys.'''
 
<source lang="python">
 
<source lang="python">
 
>>> d = {'c':22, 'b':1, 'a':10}
 
>>> d = {'c':22, 'b':1, 'a':10}
Line 119: Line 171:
 
[('a', 10), ('b',1), ('c', 22)]
 
[('a', 10), ('b',1), ('c', 22)]
 
</source>
 
</source>
==== Dictionary ====
+
'''To sort by value:'''
 +
<source lang="python">
 +
c = {'c':22, 'b':1, 'a':10}
 +
tmp = list()
 +
for k, v in c.items():
 +
    tmp.append( (v, k) )
 +
tmp.sort(reverse = True)
 +
print tmp
 +
[(22, 'c'), (10, 'a'), (1, 'b')]
 +
</source>
 +
'''The same sort by value with list comprehension'''
 +
<source lang="python">
 +
c = {'c':22, 'b':1, 'a':10}
 +
print sorted( [ (v,k) for k,v in c.items() ] )
 +
</source>
 +
<br />
 +
'''Top 10 most common words.'''
 +
<source lang="python">
 +
fhand = open('romeo.txt')
 +
counts = dict()
 +
for line in fhand:
 +
    words = line.split()
 +
    for word in words:
 +
        counts[word] = counts.get(word, 0) +1
 +
lst = list()
 +
for key, val in counts.items():
 +
    lst.append((val, key))
 +
lst.sort(reverse = True)
 +
for val, key in lst[:10]
 +
    print key, val
 +
</source>
 +
 
 +
====Dictionary====
 
Key - Value pairs. They are called diferent in diferent languages:<br />
 
Key - Value pairs. They are called diferent in diferent languages:<br />
 
dictionary[key] = value <br />
 
dictionary[key] = value <br />
  
* Perl → Associative arrays
+
*Perl → Associative arrays
* Java → Properties, Map or HashMap
+
*Java → Properties, Map or HashMap
* C# →  Property bag
+
*C# →  Property bag
 +
 
 
<br />
 
<br />
 
Declarationn and assigment:<br />
 
Declarationn and assigment:<br />
 
There are two posible ways to declare them:
 
There are two posible ways to declare them:
* purse = dict()
+
 
* puse = {}
+
*purse = dict()
 +
*puse = {}
 
<source lang="python">
 
<source lang="python">
 
purse = dict()
 
purse = dict()
Line 145: Line 231:
 
purse.get(name, default_value)
 
purse.get(name, default_value)
 
'''Other methods and functions'''<br />
 
'''Other methods and functions'''<br />
* list(purse) → Returns a list of keys.
+
 
* dict.keys() → Returns a list of keys.
+
*list(purse) → Returns a list of keys.
* dict.values() →  Returns a list of values.
+
*dict.keys() → Returns a list of keys.
* dict.items → Returns a list of tuples ( [(key, value), (key, value)...] )
+
*dict.values() →  Returns a list of values.
 +
*dict.items → Returns a list of tuples ( [(key, value), (key, value)...] )
 +
 
 
''' Word count using files and dictionarys'''<br />
 
''' Word count using files and dictionarys'''<br />
 
<source lang="python">
 
<source lang="python">
Line 170: Line 258:
 
</source>
 
</source>
  
== Conditional ==
+
==Conditional==
=== if/elif/else ===
+
===if/elif/else===
 
<source lang="python">
 
<source lang="python">
 
if a < 10:
 
if a < 10:
Line 181: Line 269:
 
</source>
 
</source>
  
=== try/except/finally ===
+
===try/except/finally===
 
<source lang="python">
 
<source lang="python">
 
try:
 
try:
Line 191: Line 279:
 
</source>
 
</source>
  
== Loops ==
+
===Print Exception===
=== For ===
+
<syntaxhighlight lang="python3">
 +
import traceback
 +
 
 +
def bug():
 +
    my_dict = {'a': 9}
 +
    try:
 +
        return my_dict['b']
 +
    except KeyError:
 +
        print(traceback.format_exc())
 +
        return None
 +
</syntaxhighlight><br />
 +
==Loops==
 +
===For===
 
<source lang="python">
 
<source lang="python">
 
NumberList = [1, 3, 7, 12, 24]
 
NumberList = [1, 3, 7, 12, 24]
Line 204: Line 304:
 
</source>
 
</source>
  
=== while ===
+
===while===
 
<source lang="python">
 
<source lang="python">
 
CtrlNum = 7
 
CtrlNum = 7
Line 212: Line 312:
 
</source>
 
</source>
  
== Functions ==
+
==Functions==
 
All arguments in Python are passed by reference, if you change a variable value inside a function it will be changed at the calling function.
 
All arguments in Python are passed by reference, if you change a variable value inside a function it will be changed at the calling function.
 
<source lang="python">
 
<source lang="python">
Line 222: Line 322:
 
         print arg
 
         print arg
 
</source>
 
</source>
 +
==Regular Expressions==
 +
<nowiki>
 +
            ^    → Matches the beginning of a line
 +
            $    → Matches the end of a line
 +
            .    → Matches any character
 +
            \s    → Matches any whitespace
 +
            \S    → Matches any non-whitespace
 +
            *    → Repeats a character 0 or more times
 +
            *?    → Repeats a character 0 or more times (non-greedy)
 +
            +      → Repeats a character 1 or more times
 +
            +?    → Repeats a character 1 or more times (non-greedy)
 +
          [aeiou] → Matches a single character in the listed set
 +
          [^XYZ]  → Matches a single character NOT in the listed set
 +
          [a-z0-9] → The set of characters can include a range
 +
            (    → Indicates where string extraction is to start
 +
            )    → Indicates where string extraction is to end</nowiki>
 +
  \    → Escape character
 +
===Regular Expression Module===
 +
It must be imported at the begining of a program:
 +
<source lang="python">
 +
import re
 +
</source>
 +
 +
*re.search(re_string, string) → similar to find()
 +
*re.findall(re_string, string) → similar to find, returns a list
 +
*re.match(re_string, string)
 +
 +
<source lang="python">
 +
import re
 +
hand = open('mbox-short.txt')
 +
for line in hand:
 +
    line = line.rstrip()
 +
    if re.search('^From:', line);
 +
        print line
 +
</source>
 +
==Date==
 +
===Today as string===
 +
<source lang="python">
 +
import datetime
 +
datetime.date.today().strftime("%B %d, %Y")
 +
</source>
 +
===String date to date type===
 +
<source lang="python">
 +
from datetime import datetime
 +
fecha_string = '2014-01-15 11:20:05.220'
 +
date = datetime.strptime(fecha_string, '%Y-%m-%d %H:%M:%S.%f')
 +
</source>
 +
 +
==Files==
 +
'''Counting lines in a file'''
 +
<source lang="python">
 +
fhand = open("words.txt")
 +
count = 0
 +
for line in fhand:
 +
    count = count + 1
 +
print "line count", count
 +
</source>
 +
'''Reading the whole file'''
 +
<source lang="python">
 +
fhand =  open("words.txt")
 +
inp = fhand.read() # reads the whole file into memory
 +
print len(inp)# returns the number of characters in a file
 +
print inp # prints the whole file
 +
</source>
 +
'''Searching through a file'''
 +
<source lang="python">
 +
fhand= open("mbox-short.txt")
 +
for line in fhand:
 +
    # line = line.rstrip()
 +
    if line.startswith("From:"):
 +
        line = line.rstrip() # better here
 +
        print line
 +
</source>
 +
'''File names and paths'''
 +
<source lang="python">
 +
fname = raw_input("Enter a file name: ")
 +
    if fname[0:2] == "~/": #Check to see if it starts with a ~ and a slash
 +
        #If it doesn't start with the ~/, then
 +
        #the user could be referring to a valid file
 +
        #like "~.py" (I checked: it is possible.)
 +
        #notice below replace is valid on Mac OSX only (and not a good approach overall, cause not portable at all)
 +
        fname = fname.replace('~',"/Users/"+raw_input("Enter your short user name: "),1)
 +
workingfname = fname.replace("\\",'') #This for proper escaping of a valid folder named '~' as '\~', you can also use './~' as Python automatically escapes for you.
 +
#go back to normal program now
 +
handle = open(workingfname,'r') # . . .
 +
for line in handle:
 +
    print line
 +
print "\n"+("That was "+fname+".").center(40)
 +
</source>
 +
===CSV===
 +
<source lang="python">
 +
import csv
 +
 +
def createCSV(simpleProducts, configProducts, shopName):
 +
    '''
 +
    Remaps a CSV filce created with tiendalistas magento exporter to
 +
    match the Shopify CSV import file
 +
    '''
 +
 +
    counter = 0
 +
    with open(OUTPUT_DIALETCT_TEMPLATE, 'rb') as csvDialectTemplate:
 +
        exportDialect = csv.Sniffer().sniff(csvDialectTemplate.read(1024))
 +
 +
    # Create file and write headers
 +
    with open(OUTPUT_PATH + shopName + '_shopify.csv', 'w') as csvoutFile:
 +
        csvWriter = csv.writer(csvoutFile, dialect=exportDialect)
 +
        csvWriter.writerow(shopyheader)
 +
        counter += 1
 +
 +
        # Write lines for simple products
 +
        for prod in simpleProducts:
 +
            tmp = shopifyCSVitem(prod)
 +
            csvWriter.writerow(tmp.productCSVline())
 +
            counter += 1
 +
 +
            for subP in prod.subProducts:
 +
                    tmp2 = shopifyCSVitem(subP)
 +
                    csvWriter.writerow(tmp2.subProductCSVline())
 +
                    counter += 1
 +
 +
        # Write lines for configurable products
 +
        for prod in configProducts:
 +
            tmp1 = shopifyCSVitem(prod)
 +
            csvWriter.writerow(tmp1.productCSVline())
 +
            counter += 1
 +
            logWrite(logFile, 'Configurable product should be reviewed', tmp1.productCSVline()[0])
 +
 +
            for subP in prod.subProducts:
 +
                tmp2 = shopifyCSVitem(subP)
 +
                csvWriter.writerow(tmp2.productCSVline())
 +
                counter += 1
 +
 +
    return counter
 +
</source>
 +
 +
==Run System Commands==
 +
 +
===Python >= 3.5===
 +
<syntaxhighlight lang="python3">
 +
def _run_command(command):
 +
    log.debug("Command: {}".format(command))
 +
    result = subprocess.run(command, shell=True, capture_output=True)
 +
 +
    if result.stderr:
 +
        raise subprocess.CalledProcessError(
 +
            returncode=result.returncode,
 +
            cmd=result.args,
 +
            stderr=f"{result.stdout}\n{result.stderr}"
 +
        )
 +
    if result.stdout:
 +
        log.debug("Command Result: {}".format(result.stdout.decode('utf-8')))
 +
    return result
 +
</syntaxhighlight>
 +
 +
==Class==
 +
<source lang="python">
 +
class MyClass:
 +
    __init__(self, arg1, arg2):
 +
        #This functionn will be executed every time you instantiate an element of this class
 +
        self.var1 = arg1
 +
        self.var2 = arg2
 +
   
 +
    def myFunction(self, param1):
 +
        self.var1 = self.var1 +param1
 +
</source>
 +
 +
Built in attributes:<br />
 +
__dict__ → Dictionary containing the class's namespace<br />
 +
__doc__ → Class documentation string or None if undefined<br />
 +
__name__ → Class name<br />
 +
__module__ → Module name in which the class is defined. This attribute is "__main__" in interactive mode.<br />
 +
__bases__ → A possibly empty tuple containing the base classes in the order of their occurrence<br />
 +
 +
'''Inheritance'''
 +
<source lang="python">
 +
class SubClassName(ParentClass1[, ParentClass2, ...]):
 +
    'Optional class documentation string'
 +
    # Class code
 +
</source>
 +
 +
*issubclass(sub, sup)
 +
*isinstance(obj, Class)
 +
 +
'''Generic functionality that can be overriden in own classes'''<br />
 +
__init__(self[, args...]) → Constructor<br />
 +
__del__(self) → Executed when Python's garbage collector destroys an object<br />
 +
__repr__(self) → Evaluable string representation<br />
 +
__str__(self) → Printable string representation<br />
 +
__cmp__(self, x) → Object comparison<br />
 +
__add__(self, other) → To define the + operator behave<br />
 +
 +
'''Hiding attributes'''
 +
attributes that start with a __ wont be visible to others<br />
 +
 +
===Subclasing builtins===
 +
 +
====Perfect dict subclass====
 +
<syntaxhighlight lang="python3">
 +
# has a ton of errors: https://stackoverflow.com/questions/3387691/how-to-perfectly-override-a-dict
 +
 +
class LowerDict(dict):
 +
    __slots__ = ()
 +
 +
    def _process_args(mapping=(), **kwargs):
 +
        if hasattr(mapping, items):
 +
            mapping = getattr(mapping, items)()
 +
        return ((ensure_lower(k), v) for k, v in chain(mapping, getattr(kwargs, items)()))
 +
 +
    def __init__(self, mapping=(), **kwargs):
 +
        super(LowerDict, self).__init__(self._process_args(mapping, **kwargs))
 +
 +
    def __getitem__(self, k):
 +
        return super(LowerDict, self).__getitem__(ensure_lower(k))
 +
    def __setitem__(self, k, v):
 +
        return super(LowerDict, self).__setitem__(ensure_lower(k), v)
 +
 +
    def __delitem__(self, k):
 +
        return super(LowerDict, self).__delitem__(ensure_lower(k))
 +
 +
    def get(self, k, default=None):
 +
        return super(LowerDict, self).get(ensure_lower(k), default)
 +
 +
    def setdefault(self, k, default=None):
 +
        return super(LowerDict, self).setdefault(ensure_lower(k), default)
 +
 +
    def pop(self, k, v=_RaiseKeyError):
 +
        if v is _RaiseKeyError:
 +
            return super(LowerDict, self).pop(ensure_lower(k))
 +
        return super(LowerDict, self).pop(ensure_lower(k), v)
 +
 +
    def update(self, mapping=(), **kwargs):
 +
        super(LowerDict, self).update(self._process_args(mapping, **kwargs))
 +
    def __contains__(self, k):
 +
        return super(LowerDict, self).__contains__(ensure_lower(k))
 +
 +
    def copy(self): # don't delegate w/ super - dict.copy() -> dict :(
 +
        return type(self)(self)
 +
 +
    @classmethod
 +
    def fromkeys(cls, keys, v=None):
 +
        return super(LowerDict, cls).fromkeys((ensure_lower(k) for k in keys), v)
 +
 +
    def __repr__(self):
 +
        return '{0}({1})'.format(type(self).__name__, super(LowerDict, self).__repr__())
 +
</syntaxhighlight>
 +
 +
====Not so perfect dict like object====
 +
<syntaxhighlight lang="python3">
 +
from collections.abc import MutableMapping
 +
 +
 +
class TransformedDict(MutableMapping):
 +
    """A dictionary that applies an arbitrary key-altering
 +
      function before accessing the keys"""
 +
 +
    def __init__(self, *args, **kwargs):
 +
        self.store = dict()
 +
        self.update(dict(*args, **kwargs))  # use the free update to set keys
 +
 +
    def __getitem__(self, key):
 +
        return self.store[self._keytransform(key)]
 +
 +
    def __setitem__(self, key, value):
 +
        self.store[self._keytransform(key)] = value
 +
 +
    def __delitem__(self, key):
 +
        del self.store[self._keytransform(key)]
 +
 +
    def __iter__(self):
 +
        return iter(self.store)
 +
   
 +
    def __len__(self):
 +
        return len(self.store)
 +
 +
    def _keytransform(self, key):
 +
        return key
 +
</syntaxhighlight>
 +
 +
==os.system()==
 +
To execute a Linux command from Python:
 +
<source lang="python">
 +
os.system(command) → returns exit status
 +
</source>
 +
==Database Access==
 +
===MySQLdb===
 +
'''Table creation'''
 +
<source lang="python">
 +
#!/usr/bin/python
 +
 +
import MySQLdb
 +
 +
# Open database connection
 +
db = MySQLdb.connect("DB_HOST","DB_USER","DB_PASSWORD","DB_NAME" )
  
== Files ==
+
# prepare a cursor object using cursor() method
== Class ==
+
cursor = db.cursor()
== MySQLdb ==
+
 
== os.system() ==
+
# Drop table if it already exist using execute() method.
 +
cursor.execute("DROP TABLE IF EXISTS EMPLOYEE")
 +
 
 +
# Create table as per requirement
 +
sql = """CREATE TABLE EMPLOYEE (
 +
        FIRST_NAME  CHAR(20) NOT NULL,
 +
        LAST_NAME  CHAR(20),
 +
        AGE INT, 
 +
        SEX CHAR(1),
 +
        INCOME FLOAT )"""
 +
 
 +
cursor.execute(sql)
 +
</source>
 +
'''INSERT operation'''
 +
<source lang="python">
 +
#!/usr/bin/python
 +
 
 +
import MySQLdb
 +
 
 +
# Open database connection
 +
db = MySQLdb.connect("DB_HOST","DB_USER","DB_PASSWORD","DB_NAME" )
 +
 
 +
# prepare a cursor object using cursor() method
 +
cursor = db.cursor()
 +
 
 +
# Prepare SQL query to INSERT a record into the database.
 +
sql = "INSERT INTO EMPLOYEE(FIRST_NAME, \
 +
      LAST_NAME, AGE, SEX, INCOME) \
 +
      VALUES ('%s', '%s', '%d', '%c', '%d' )" % \
 +
      ('Mac', 'Mohan', 20, 'M', 2000)
 +
try:
 +
  # Execute the SQL command
 +
  cursor.execute(sql)
 +
  # Commit your changes in the database
 +
  db.commit()
 +
except:
 +
  # Rollback in case there is any error
 +
  db.rollback()
 +
 
 +
# disconnect from server
 +
db.close()
 +
</source>
 +
 
 +
'''Read example'''
 +
Once the query is made you can:
 +
 
 +
*fetchone()
 +
*fetchall()
 +
*rowcount
 +
<source lang="python">
 +
#!/usr/bin/python
 +
 
 +
import MySQLdb
 +
 
 +
# Open database connection
 +
db = MySQLdb.connect("localhost","testuser","test123","TESTDB" )
 +
 
 +
# prepare a cursor object using cursor() method
 +
cursor = db.cursor()
 +
 
 +
# Prepare SQL query to INSERT a record into the database.
 +
sql = "SELECT * FROM EMPLOYEE \
 +
      WHERE INCOME > '%d'" % (1000)
 +
try:
 +
  # Execute the SQL command
 +
  cursor.execute(sql)
 +
  # Fetch all the rows in a list of lists.
 +
  results = cursor.fetchall()
 +
  print "Found ', cursor.rowcount, ' results: '
 +
  for row in results:
 +
      fname = row[0]
 +
      lname = row[1]
 +
      age = row[2]
 +
      sex = row[3]
 +
      income = row[4]
 +
      # Now print fetched result
 +
      print "fname=%s,lname=%s,age=%d,sex=%s,income=%d" % \
 +
            (fname, lname, age, sex, income )
 +
except:
 +
  print "Error: unable to fecth data"
 +
 
 +
# disconnect from server
 +
db.close()
 +
</source>
 +
===PostgreSQL===
 +
Refer to: [https://wiki.python.org/moin/PostgreSQL PostgreSQL]
 +
===Other===
 +
Refer to: [https://wiki.python.org/moin/DatabaseInterfaces Database Interfaces]
 +
 
 +
==ssh tunneling==
 +
<source lang="python">
 +
import subprocess
 +
sshcommand = ['ssh', '-o', 'StrictHostKeyChecking=no', '-i', keyFilePath, '-N', '-L', '8888:localhost:3306', 'user@' + Host]
 +
ssh_proc = subprocess.Popen(sshcommand, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
 +
# Do stuff
 +
ssh_proc.terminate()
 +
</source>
 +
 
 +
==Multithreading==
 +
<source lang="python">
 +
#!/usr/bin/python
 +
 
 +
import thread
 +
import time
 +
 
 +
#Define a function for the thread
 +
def print_time( threadName, delay):
 +
  count = 0
 +
  while count < 5:
 +
      time.sleep(delay)
 +
      count += 1
 +
      print "%s: %s" % ( threadName, time.ctime(time.time()) )
 +
 
 +
# Create two threads as follows
 +
try:
 +
  thread.start_new_thread( print_time, ("Thread-1", 2, ) )
 +
  thread.start_new_thread( print_time, ("Thread-2", 4, ) )
 +
except:
 +
  print "Error: unable to start thread"
 +
 
 +
while 1:
 +
  pass
 +
</source>
 +
(*) see also threading module:
 +
<source lang="python">
 +
import threading
 +
</source>
 +
==Python GUI Programming==
 +
 
 +
*Tkinter: Tkinter is the Python interface to the Tk GUI toolkit shipped with Python. We would look this option in this tutorial.
 +
*wxPython: This is an open-source Python interface for wxWindows http://wxpython.org.
 +
*JPython: JPython is a Python port for Java which gives Python scripts seamless access to Java class libraries on the local machine http://www.jython.org.
 +
 
 +
'''Tkinter'''
 +
<source lang="python">
 +
#!/usr/bin/python
 +
 
 +
import Tkinter
 +
top = Tkinter.Tk()
 +
# Code to add widgets will go here...
 +
top.mainloop()
 +
</source>
 +
'''Widgets'''
 +
 
 +
*Button The Button widget is used to display buttons in your application.
 +
*Canvas The Canvas widget is used to draw shapes, such as lines, ovals, polygons and rectangles, in your application.
 +
*Checkbutton The Checkbutton widget is used to display a number of options as checkboxes. The user can select multiple options at a time.
 +
*Entry The Entry widget is used to display a single-line text field for accepting values from a user.
 +
*Frame The Frame widget is used as a container widget to organize other widgets.
 +
*Label The Label widget is used to provide a single-line caption for other widgets. It can also contain images.
 +
*Listbox The Listbox widget is used to provide a list of options to a user.
 +
*Menubutton The Menubutton widget is used to display menus in your application.
 +
*Menu The Menu widget is used to provide various commands to a user. These commands are contained inside Menubutton.
 +
*Message The Message widget is used to display multiline text fields for accepting values from a user.
 +
*Radiobutton The Radiobutton widget is used to display a number of options as radio buttons. The user can select only one option at a time.
 +
*Scale The Scale widget is used to provide a slider widget.
 +
*Scrollbar The Scrollbar widget is used to add scrolling capability to various widgets, such as list boxes.
 +
*Text The Text widget is used to display text in multiple lines.
 +
*Toplevel The Toplevel widget is used to provide a separate window container.
 +
*Spinbox The Spinbox widget is a variant of the standard Tkinter Entry widget, which can be used to select from a fixed number of values.
 +
*PanedWindow A PanedWindow is a container widget that may contain any number of panes, arranged horizontally or vertically.
 +
*LabelFrame A labelframe is a simple container widget. Its primary purpose is to act as a spacer or container for complex window layouts.
 +
*tkMessageBox This module is used to display message boxes in your applications.
 +
 
 +
'''Standard attributes'''
 +
 
 +
*Dimensions
 +
*Colors
 +
*Fonts
 +
*Anchors
 +
*Relief styles
 +
*Bitmaps
 +
*Cursors
 +
 
 +
'''Geometry Management'''
 +
 
 +
*The pack() Method - This geometry manager organizes widgets in blocks before placing them in the parent widget.
 +
*The grid() Method - This geometry manager organizes widgets in a table-like structure in the parent widget.
 +
*The place() Method -This geometry manager organizes widgets by placing them in a specific position in the parent widget.
 +
 
 +
For further reference visit: tutorialspoint.com/python/python_gui_programming.htm
 +
 
 +
==Enable auto complete in python interpreter==
 +
Create a file in your home directory named: .pythonrc<br />
 +
Content of this file:
 +
<source lang="python">
 +
import rlcompleter, readline
 +
readline.parse_and_bind('tab:complete')
 +
</source>
 +
Set the PYTHONSTARTUP variable in your .bashrc or .bash_profile
 +
echo "export PYTHONSTARTUP=~/.pythonrc" &gt;&gt; .bashrc
 +
 
 +
Reload your .bashrc or .bash_profile
 +
source ~/.bashrc
 +
 
 +
To test it import a library, write librariname. and hit tab twice
 +
 
 +
==Delete *.pyc==
 +
find . -name "*.pyc" -exec rm -rf {} \;
 +
 
 +
==format hex==
 +
 
 +
 
 +
Use the format() function with a '02x' format.
 +
 
 +
>>> format(255, '02x')<br />
 +
'ff'<br />
 +
>>> format(2, '02x')<br />
 +
'02'<br />
 +
 
 +
The 02 part tells format() to use at least 2 digits and to use zeros to pad it to length, x <br />means lower-case hexadecimal.<br />
 +
 
 +
The Format Specification Mini Language also gives you X for uppercase hex output, and you can <br />prefix the field width with # to include a 0x or 0X prefix (depending on wether you used <br />x or X as the formatter). Just take into account that you need to adjust the field width <br />to allow for those extra 2 characters:<br />
 +
 
 +
>>> format(255, '02X')<br />
 +
'FF'<br />
 +
>>> format(255, '#04x')<br />
 +
'0xff'<br />
 +
>>> format(255, '#04X')<br />
 +
'0XFF'<br />
 +
 
 +
==Command completion==
 +
$ nano ~/.pythonrc
 +
 
 +
<source lang="python">
 +
# ~/.pythonrc
 +
# enable syntax completion
 +
try:
 +
    import readline
 +
except ImportError:
 +
    print("Module readline not available.")
 +
else:
 +
    import rlcompleter
 +
    readline.parse_and_bind("tab: complete")
 +
</source>
 +
 
 +
$ nano ~/.bashrc
 +
<source lang="bash">
 +
export PYTHONSTARTUP=~/.pythonrc</source>
 +
 
 +
==nose debug==
 +
nosetests --debug=nose,nose.importer --debug-log=nose_debug <your usual args>
 +
 
 +
==Publish Python Package==
 +
Upload a package to pypi. This would make your package available with:
 +
pip install mypackage
 +
===With setup.py===
 +
nano ~/.pypirc
 +
<nowiki>[distutils]
 +
          index-servers =
 +
            pypi
 +
            pypitest
 +
         
 +
          [pypi]
 +
          repository=https://upload.pypi.org/legacy/
 +
          username=your_username
 +
          password=your_password
 +
         
 +
          [pypitest]
 +
          repository=https://testpypi.python.org/pypi
 +
          username=your_username
 +
          password=your_password</nowiki>
 +
Adjust .pypirc permissions
 +
chmod 600 ~/.pypirc
 +
In your project, create a setup.py
 +
<source lang="python">import os
 +
from setuptools import setup, find_packages
 +
 
 +
# from distutils.core import setup
 +
 
 +
# allow setup.py to be run from any path
 +
os.chdir(os.path.normpath(os.path.join(os.path.abspath(__file__), os.pardir)))
 +
 
 +
 
 +
def here(name):
 +
    return os.path.join(
 +
        os.path.dirname(os.path.abspath(__file__)),
 +
        name)
 +
 
 +
 
 +
def read(name, mode='rb', encoding='utf8'):
 +
    os.system('pandoc --from=markdown --to=rst --output=README.rst README.md')
 +
    if os.path.exists('README.rst'):
 +
        long_description = open('README.rst').read()
 +
    else:
 +
        try:
 +
            with open(here(name), mode) as fp:
 +
                long_description = fp.read()
 +
        except IOError:
 +
            return 'Error generating long description: {} File not found'.format(here(name))
 +
    return long_description
 +
 
 +
# Development Status :: 1 - Planning
 +
# Development Status :: 2 - Pre-Alpha
 +
# Development Status :: 3 - Alpha
 +
# Development Status :: 4 - Beta
 +
# Development Status :: 5 - Production/Stable
 +
# Development Status :: 6 - Mature
 +
# Development Status :: 7 - Inactive
 +
 
 +
 
 +
license_classifiers = {
 +
    'MIT license': 'License :: OSI Approved :: MIT License',
 +
    'BSD license': 'License :: OSI Approved :: BSD License',
 +
    'ISC license': 'License :: OSI Approved :: ISC License (ISCL)',
 +
    'Apache Software License 2.0': 'License :: OSI Approved :: Apache Software License',
 +
    'GNU General Public License v3': 'License :: OSI Approved :: GNU General Public License v3 (GPLv3)'
 +
}
 +
 
 +
setup(
 +
    name='CheKnife',
 +
    version='v0.0.6b2',
 +
    packages=find_packages(),
 +
    url='https://git.herrerosolis.com/Misc/CheKnife',
 +
    download_url='https://git.herrerosolis.com/Misc/CheKnife/-/archive/v0.0.6b2/CheKnife-v0.0.6b2.tar.gz',
 +
    license='MIT license',
 +
    author='Rafael Herrero Solis',
 +
    author_email='rafahsolis@hotmail.com',
 +
    keywords=['CheKnife', 'Swiss', 'Army', 'Knife', 'Swiss Army Knife'],
 +
    description='Python Utilities',
 +
    long_description=read('README.md'),
 +
    test_suite='nose.collector',
 +
    tests_require=['nose', 'six'],
 +
    install_requires=[
 +
        'six>=1.10.0',
 +
        'future>=0.16.0',
 +
        'pycryptodome>=3.6.1',
 +
        'configparser>=3.5.0'
 +
    ],
 +
    classifiers=[
 +
        'License :: OSI Approved :: MIT License',
 +
        'Development Status :: 3 - Alpha',
 +
        'License :: OSI Approved :: Apache Software License',
 +
        'Programming Language :: Python :: 3.6',
 +
    ],
 +
)</source>
 +
 
 +
nano setup.cfg
 +
<nowiki>[metadata]
 +
          description-file = README.md</nowiki>
 +
 
 +
nano LICENSE
 +
<nowiki>The MIT License
 +
         
 +
          SPDX short identifier: MIT
 +
         
 +
          Copyright <YEAR> <COPYRIGHT HOLDER>
 +
         
 +
          Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
 +
         
 +
          The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
 +
         
 +
          THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.</nowiki>
 +
Test and upload
 +
<source lang="bash">python setup.py register -r pypitest
 +
python setup.py sdist upload -r pypitest
 +
python setup.py register -r pypi
 +
python setup.py sdist upload -r pypi</source>
 +
 
 +
===With twine===
 +
https://packaging.python.org/tutorials/packaging-projects/
 +
 
 +
==Change PyPi==
 +
===Using PyPi Local Repository===
 +
nano ~/.pip/pip.conf
 +
<nowiki>[global]
 +
          index-url= http://10.255.0.21/pypi/simple
 +
          trusted-host= 10.255.0.21</nowiki>
 +
 
 +
==Using pip behind proxy==
 +
sudo pip --proxy http://proxy.hell:3128 install requests
 +
 
 +
==Publish to PyPi==
 +
Create the file ~/.pypirc
 +
<nowiki>[distutils]
 +
          index-servers =
 +
              pypi
 +
              pypitest
 +
         
 +
          [pypi]
 +
              repository=https://pypi.python.org/pypi
 +
              username=
 +
              password=
 +
         
 +
          [pypitest]
 +
              repository=https://test.pypi.org/legacy
 +
              username=
 +
              password=
 +
          </nowiki>
 +
 
 +
Test
 +
python setup.py register -r pypitest
 +
python setup.py sdist upload -r pypitest
 +
 
 +
Upload
 +
python setup.py register -r pypi
 +
python setup.py sdist upload -r pypi
 +
 
 +
== requirements.txt ==
 +
 
 +
=== Platform conditional ===
 +
<syntaxhighlight lang="text">
 +
psycopg2-binary~=2.9.7; platform_system == "Linux"
 +
psycopg2~=2.9.6; platform_system == "Windows"
 +
</syntaxhighlight>
 +
 
 +
==AWS boto3 EC2==
 +
<source lang="python">
 +
import boto3
 +
client = boto3.client('ec2', aws_access_key_id="AKI.............Q", aws_secret_access_key="2..........................s", region_name='us-east-1')
 +
ec2_regions = [region['RegionName'] for region in client.describe_regions()['Regions']]
 +
instances = []
 +
for region in ec2_regions:
 +
    conn = boto3.resource('ec2', aws_access_key_id="AKIAJBWPORUPPGNSUWXQ", aws_secret_access_key="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", region_name=region)
 +
    instances += conn.instances.filter()
 +
i = instances[0]
 +
i.public_dns_name
 +
i.public_ip_address
 +
</source>
 +
[[Category:Python]]

Latest revision as of 08:08, 11 September 2024

Install

Indentation

Python is an indented language, so the code indentation matters. A good practice is to indent with 4 spaces (if you mix spaces and tabs the code won't work.

Operators

Arithmetic

+    Addition
          -    Substraction
          *    Multiplication
          /    Division
          **   Power
          %    Reminder
          //   Floor division (Reminder is removed)

Comparison

          ==   Equal 
          !=   Not equal
          >    Greater
          >=   Greater or equal
          <    Smaller
          <=   Smaller or equal
          <>   Similar to !=

Assignment

          =    Simple assignment
          +=   Add and as
          -=   Substract and  assignment
          *=   Multiply and assignment
          /=   Divide and assignment
          %=   Modulus and assignment
          **=  Exponent and assignment
          //=  Floor Divisionn and assignment

Bitwise operators

They perform operations on binary terms. a= 8 → 100; b= 9 → 101; a & b → 100

          $    Binary AND
          |    Binary OR
          ^    Binary XOR
          ~    Binary complement
          <<   Binary left shift
          >>   Binary right shift

Logic

          and
          or
          not

Membership operators

in
not in

Variables

Numbers

0 padding 2 digits

print("{:02d}".format(1))

String

String assingment:
MyString = 'Hello World' OR MyString = "Hello World"
Strings can be subset sliced:

MyString = "Hello world"
MyString[0:4]

Hel

in operator: to check if a substring is contained in a string

String library:

str.lower()
str.upper()
str.capitalize()  Uppercases 1st char
str.center(width[, fillchar])
str.startswith(preffix[, start[, end]])
str.endswith(suffix[, start[, end]])
str.find(substring[, start[, end]])
str.lstrip([chars])
str.rstrip([chars])
str.strip([chars])
str.replace(old, new[, count])see also threading module:

import threading


str.split([char])

len(MyString) → returns the lenght of a string

Collections

List

Collection - allows us to put many values in a single variable. Simple variables are not collections. The previous value is overwritten when changed.

  • A List is made up of list 'constants'. Lists are surrounded by square brackets [] and the constants in the list are separated by commas. ([2,4,6,8])
  • A List element can be any Python object, even another list
  • A List can be empty
  • Lists are mutable (they can be changed)
  • When len() is used on a list, it counts the number of constants that make up the list. (not the number of characters)
  • Lists can be concatenated using +
  • Lists can be sliced
  • List is a unique type that can be checked using type() (result: <type 'list'>)
  • An empty list can be created with list()
  • Lists can be tested for contents using in/not in
  • List is an ordered sequence
  • A list can be sorted with .sort(). Sort changes the list permanently.
  • sorted(list) → Returns a list sorted.
  • Methods: append, count, extend, index, insert, pop, remove, reverse, sort
  • Functions len() - find length, max() - find highest value, min() - find lowest value, sum() - add all values average can be found with sum()/len()
  • reversed list:
    • list.reverse()
    • list[::-1]
    • reversed(list)

Del Removes an item from a list.
example: a = [1, 2, 3, 4, 5] del a[0] returns [2, 3, 4, 5]


List comprehension

flatten list

def flatten(list_of_lists):
    return [element for secondary_list in list_of_lists for element in secondary_list ]

a = [[1,2,3], [3,4,5], ["a", "b"]]
flatten(a)

Out[29]: [1, 2, 3, 3, 4, 5, 'a', 'b']


Tuple

Inmutable, declared by ()

  • My_tuple = (3, 2, 1)

You can iterate trough tuples.
You can also refer to the items of a tuple like in a list.

My_tuple[0]
3


They support multple assignment, usefull for example to iterate trough dictionaries:

d = dict()
d['steve'] = 2
d['susan'] = 4

for (k, v) in d.items():
    print k, v

Tuples are comparable. They compare the leftmost first, then the next and so on...

>>> (0, 1, 2) < (5, 1, 2)
True
>>> (0, 1, 30000000) < (0, 3, 1)
True

To sort dictionaries by keys.

>>> d = {'c':22, 'b':1, 'a':10}
>>> t = d.items()
>>> t
[('c', 22), ('b',1), ('a', 10)]
>>> t.sort()
>>> t
[('a', 10), ('b',1), ('c', 22)]

To sort by value:

c = {'c':22, 'b':1, 'a':10}
tmp = list()
for k, v in c.items():
    tmp.append( (v, k) )
tmp.sort(reverse = True)
print tmp
[(22, 'c'), (10, 'a'), (1, 'b')]

The same sort by value with list comprehension

c = {'c':22, 'b':1, 'a':10}
print sorted( [ (v,k) for k,v in c.items() ] )


Top 10 most common words.

fhand = open('romeo.txt')
counts = dict()
for line in fhand:
    words = line.split()
    for word in words:
        counts[word] = counts.get(word, 0) +1
lst = list()
for key, val in counts.items():
    lst.append((val, key))
lst.sort(reverse = True)
for val, key in lst[:10]
    print key, val

Dictionary

Key - Value pairs. They are called diferent in diferent languages:
dictionary[key] = value

  • Perl → Associative arrays
  • Java → Properties, Map or HashMap
  • C# → Property bag


Declarationn and assigment:
There are two posible ways to declare them:

  • purse = dict()
  • puse = {}
purse = dict()
purse['money'] = 12
purse['candy'] = 3

print purse['money']

purse = {'money': 12, 'candy' = 3}

get() method:
If you try to retrieve a value for a key that doesn't exist you would get a traceback error.
To avoid this you should use the get() method, that returns the default value if the key doesn't exist: purse.get(name, default_value) Other methods and functions

  • list(purse) → Returns a list of keys.
  • dict.keys() → Returns a list of keys.
  • dict.values() → Returns a list of values.
  • dict.items → Returns a list of tuples ( [(key, value), (key, value)...] )

Word count using files and dictionarys

counts = dict()
file_path = raw_input('Enter file name: ')
file_handle = open(file_path, 'r')
text = file_handle.read()
words = text.split()

for word in words:
    counts[word] = counts.get(word, 0) + 1

bigCount = None
bigWord  = None
for word, count in counts.items():
    if bigCount is None or count > bigCount:
        bigWord = word
        bigCount = count

print 'Most frequent word: ', bigWord, '\nFrequency: ', bigCount'

Conditional

if/elif/else

if a < 10:
    print "Less than 10"
elif a >= 10 and a < 20:
    print "a greater or equal to 10 and less than 20"
else:
    print "a greater or equal to 20"

try/except/finally

try:
    file = open("test.txt")
except:
    print "Could not open file"
finally:
    print "This part will be executed at the end whether the open fails or not"

Print Exception

import traceback

def bug():
    my_dict = {'a': 9}
    try:
        return my_dict['b']
    except KeyError:
        print(traceback.format_exc())
        return None


Loops

For

NumberList = [1, 3, 7, 12, 24]
for number in NumberList:
    print number

Another way:

for i in range(0, len(NumberList)-1):
    print NumberList[i]

while

CtrlNum = 7
while CtrllNum > 3:
    print CtrlNum
    CtrlNum -= 1

Functions

All arguments in Python are passed by reference, if you change a variable value inside a function it will be changed at the calling function.

def MyFunction(formal_arg, optional_arg = None, *variable_lenght_args):
    print formal_arg
    if optional_arg:
        print optional_arg
    for arg in variable_lenght_args:
        print arg

Regular Expressions

             ^     → Matches the beginning of a line
             $     → Matches the end of a line
             .     → Matches any character
            \s     → Matches any whitespace
            \S     → Matches any non-whitespace
             *     → Repeats a character 0 or more times
            *?     → Repeats a character 0 or more times (non-greedy)
            +      → Repeats a character 1 or more times
            +?     → Repeats a character 1 or more times (non-greedy)
           [aeiou] → Matches a single character in the listed set
           [^XYZ]  → Matches a single character NOT in the listed set
          [a-z0-9] → The set of characters can include a range
             (     → Indicates where string extraction is to start
             )     → Indicates where string extraction is to end 
  \     → Escape character

Regular Expression Module

It must be imported at the begining of a program:

import re
  • re.search(re_string, string) → similar to find()
  • re.findall(re_string, string) → similar to find, returns a list
  • re.match(re_string, string)
import re
hand = open('mbox-short.txt')
for line in hand:
    line = line.rstrip()
    if re.search('^From:', line);
        print line

Date

Today as string

import datetime
datetime.date.today().strftime("%B %d, %Y")

String date to date type

from datetime import datetime
fecha_string = '2014-01-15 11:20:05.220'
date = datetime.strptime(fecha_string, '%Y-%m-%d %H:%M:%S.%f')

Files

Counting lines in a file

fhand = open("words.txt")
count = 0
for line in fhand:
    count = count + 1
print "line count", count

Reading the whole file

fhand =  open("words.txt")
inp = fhand.read() # reads the whole file into memory
print len(inp)# returns the number of characters in a file
print inp # prints the whole file

Searching through a file

fhand= open("mbox-short.txt")
for line in fhand:
    # line = line.rstrip()
    if line.startswith("From:"):
        line = line.rstrip() # better here
        print line

File names and paths

fname = raw_input("Enter a file name: ")
    if fname[0:2] == "~/": #Check to see if it starts with a ~ and a slash
        #If it doesn't start with the ~/, then 
        #the user could be referring to a valid file
        #like "~.py" (I checked: it is possible.)
        #notice below replace is valid on Mac OSX only (and not a good approach overall, cause not portable at all)
        fname = fname.replace('~',"/Users/"+raw_input("Enter your short user name: "),1)
workingfname = fname.replace("\\",'') #This for proper escaping of a valid folder named '~' as '\~', you can also use './~' as Python automatically escapes for you.
#go back to normal program now
handle = open(workingfname,'r') # . . .
for line in handle:
    print line
print "\n"+("That was "+fname+".").center(40)

CSV

import csv

def createCSV(simpleProducts, configProducts, shopName):
    '''
    Remaps a CSV filce created with tiendalistas magento exporter to
    match the Shopify CSV import file
    '''

    counter = 0
    with open(OUTPUT_DIALETCT_TEMPLATE, 'rb') as csvDialectTemplate:
        exportDialect = csv.Sniffer().sniff(csvDialectTemplate.read(1024))

    # Create file and write headers
    with open(OUTPUT_PATH + shopName + '_shopify.csv', 'w') as csvoutFile:
        csvWriter = csv.writer(csvoutFile, dialect=exportDialect)
        csvWriter.writerow(shopyheader)
        counter += 1

        # Write lines for simple products
        for prod in simpleProducts:
            tmp = shopifyCSVitem(prod)
            csvWriter.writerow(tmp.productCSVline())
            counter += 1

            for subP in prod.subProducts:
                    tmp2 = shopifyCSVitem(subP)
                    csvWriter.writerow(tmp2.subProductCSVline())
                    counter += 1

        # Write lines for configurable products
        for prod in configProducts:
            tmp1 = shopifyCSVitem(prod)
            csvWriter.writerow(tmp1.productCSVline())
            counter += 1
            logWrite(logFile, 'Configurable product should be reviewed', tmp1.productCSVline()[0])

            for subP in prod.subProducts:
                tmp2 = shopifyCSVitem(subP)
                csvWriter.writerow(tmp2.productCSVline())
                counter += 1

    return counter

Run System Commands

Python >= 3.5

def _run_command(command):
    log.debug("Command: {}".format(command))
    result = subprocess.run(command, shell=True, capture_output=True)

    if result.stderr:
        raise subprocess.CalledProcessError(
            returncode=result.returncode,
            cmd=result.args,
            stderr=f"{result.stdout}\n{result.stderr}"
        )
    if result.stdout:
        log.debug("Command Result: {}".format(result.stdout.decode('utf-8')))
    return result

Class

class MyClass:
    __init__(self, arg1, arg2):
        #This functionn will be executed every time you instantiate an element of this class
        self.var1 = arg1
        self.var2 = arg2
    
    def myFunction(self, param1):
        self.var1 = self.var1 +param1

Built in attributes:
__dict__ → Dictionary containing the class's namespace
__doc__ → Class documentation string or None if undefined
__name__ → Class name
__module__ → Module name in which the class is defined. This attribute is "__main__" in interactive mode.
__bases__ → A possibly empty tuple containing the base classes in the order of their occurrence

Inheritance

class SubClassName(ParentClass1[, ParentClass2, ...]):
    'Optional class documentation string'
    # Class code
  • issubclass(sub, sup)
  • isinstance(obj, Class)

Generic functionality that can be overriden in own classes
__init__(self[, args...]) → Constructor
__del__(self) → Executed when Python's garbage collector destroys an object
__repr__(self) → Evaluable string representation
__str__(self) → Printable string representation
__cmp__(self, x) → Object comparison
__add__(self, other) → To define the + operator behave

Hiding attributes attributes that start with a __ wont be visible to others

Subclasing builtins

Perfect dict subclass

# has a ton of errors: https://stackoverflow.com/questions/3387691/how-to-perfectly-override-a-dict

class LowerDict(dict):
    __slots__ = ()

    def _process_args(mapping=(), **kwargs):
        if hasattr(mapping, items):
            mapping = getattr(mapping, items)()
        return ((ensure_lower(k), v) for k, v in chain(mapping, getattr(kwargs, items)()))

    def __init__(self, mapping=(), **kwargs):
        super(LowerDict, self).__init__(self._process_args(mapping, **kwargs))

    def __getitem__(self, k):
        return super(LowerDict, self).__getitem__(ensure_lower(k))
    def __setitem__(self, k, v):
        return super(LowerDict, self).__setitem__(ensure_lower(k), v)

    def __delitem__(self, k):
        return super(LowerDict, self).__delitem__(ensure_lower(k))

    def get(self, k, default=None):
        return super(LowerDict, self).get(ensure_lower(k), default)

    def setdefault(self, k, default=None):
        return super(LowerDict, self).setdefault(ensure_lower(k), default)

    def pop(self, k, v=_RaiseKeyError):
        if v is _RaiseKeyError:
            return super(LowerDict, self).pop(ensure_lower(k))
        return super(LowerDict, self).pop(ensure_lower(k), v)

    def update(self, mapping=(), **kwargs):
        super(LowerDict, self).update(self._process_args(mapping, **kwargs))
    def __contains__(self, k):
        return super(LowerDict, self).__contains__(ensure_lower(k))

    def copy(self): # don't delegate w/ super - dict.copy() -> dict :(
        return type(self)(self)

    @classmethod
    def fromkeys(cls, keys, v=None):
        return super(LowerDict, cls).fromkeys((ensure_lower(k) for k in keys), v)

    def __repr__(self):
        return '{0}({1})'.format(type(self).__name__, super(LowerDict, self).__repr__())

Not so perfect dict like object

from collections.abc import MutableMapping


class TransformedDict(MutableMapping):
    """A dictionary that applies an arbitrary key-altering
       function before accessing the keys"""

    def __init__(self, *args, **kwargs):
        self.store = dict()
        self.update(dict(*args, **kwargs))  # use the free update to set keys

    def __getitem__(self, key):
        return self.store[self._keytransform(key)]

    def __setitem__(self, key, value):
        self.store[self._keytransform(key)] = value

    def __delitem__(self, key):
        del self.store[self._keytransform(key)]

    def __iter__(self):
        return iter(self.store)
    
    def __len__(self):
        return len(self.store)

    def _keytransform(self, key):
        return key

os.system()

To execute a Linux command from Python:

os.system(command)  returns exit status

Database Access

MySQLdb

Table creation

#!/usr/bin/python

import MySQLdb

# Open database connection
db = MySQLdb.connect("DB_HOST","DB_USER","DB_PASSWORD","DB_NAME" )

# prepare a cursor object using cursor() method
cursor = db.cursor()

# Drop table if it already exist using execute() method.
cursor.execute("DROP TABLE IF EXISTS EMPLOYEE")

# Create table as per requirement
sql = """CREATE TABLE EMPLOYEE (
         FIRST_NAME  CHAR(20) NOT NULL,
         LAST_NAME  CHAR(20),
         AGE INT,  
         SEX CHAR(1),
         INCOME FLOAT )"""

cursor.execute(sql)

INSERT operation

#!/usr/bin/python

import MySQLdb

# Open database connection
db = MySQLdb.connect("DB_HOST","DB_USER","DB_PASSWORD","DB_NAME" )

# prepare a cursor object using cursor() method
cursor = db.cursor()

# Prepare SQL query to INSERT a record into the database.
sql = "INSERT INTO EMPLOYEE(FIRST_NAME, \
       LAST_NAME, AGE, SEX, INCOME) \
       VALUES ('%s', '%s', '%d', '%c', '%d' )" % \
       ('Mac', 'Mohan', 20, 'M', 2000)
try:
   # Execute the SQL command
   cursor.execute(sql)
   # Commit your changes in the database
   db.commit()
except:
   # Rollback in case there is any error
   db.rollback()

# disconnect from server
db.close()

Read example Once the query is made you can:

  • fetchone()
  • fetchall()
  • rowcount
#!/usr/bin/python

import MySQLdb

# Open database connection
db = MySQLdb.connect("localhost","testuser","test123","TESTDB" )

# prepare a cursor object using cursor() method
cursor = db.cursor()

# Prepare SQL query to INSERT a record into the database.
sql = "SELECT * FROM EMPLOYEE \
       WHERE INCOME > '%d'" % (1000)
try:
   # Execute the SQL command
   cursor.execute(sql)
   # Fetch all the rows in a list of lists.
   results = cursor.fetchall()
   print "Found ', cursor.rowcount, ' results: ' 
   for row in results:
      fname = row[0]
      lname = row[1]
      age = row[2]
      sex = row[3]
      income = row[4]
      # Now print fetched result
      print "fname=%s,lname=%s,age=%d,sex=%s,income=%d" % \
             (fname, lname, age, sex, income )
except:
   print "Error: unable to fecth data"

# disconnect from server
db.close()

PostgreSQL

Refer to: PostgreSQL

Other

Refer to: Database Interfaces

ssh tunneling

import subprocess
sshcommand = ['ssh', '-o', 'StrictHostKeyChecking=no', '-i', keyFilePath, '-N', '-L', '8888:localhost:3306', 'user@' + Host]
ssh_proc = subprocess.Popen(sshcommand, stdin=subprocess.PIPE, stdout=subprocess.PIPE)
# Do stuff
ssh_proc.terminate()

Multithreading

#!/usr/bin/python

import thread
import time

#Define a function for the thread
def print_time( threadName, delay):
   count = 0
   while count < 5:
      time.sleep(delay)
      count += 1
      print "%s: %s" % ( threadName, time.ctime(time.time()) )

# Create two threads as follows
try:
   thread.start_new_thread( print_time, ("Thread-1", 2, ) )
   thread.start_new_thread( print_time, ("Thread-2", 4, ) )
except:
   print "Error: unable to start thread"

while 1:
   pass

(*) see also threading module:

import threading

Python GUI Programming

  • Tkinter: Tkinter is the Python interface to the Tk GUI toolkit shipped with Python. We would look this option in this tutorial.
  • wxPython: This is an open-source Python interface for wxWindows http://wxpython.org.
  • JPython: JPython is a Python port for Java which gives Python scripts seamless access to Java class libraries on the local machine http://www.jython.org.

Tkinter

#!/usr/bin/python

import Tkinter
top = Tkinter.Tk()
# Code to add widgets will go here...
top.mainloop()

Widgets

  • Button The Button widget is used to display buttons in your application.
  • Canvas The Canvas widget is used to draw shapes, such as lines, ovals, polygons and rectangles, in your application.
  • Checkbutton The Checkbutton widget is used to display a number of options as checkboxes. The user can select multiple options at a time.
  • Entry The Entry widget is used to display a single-line text field for accepting values from a user.
  • Frame The Frame widget is used as a container widget to organize other widgets.
  • Label The Label widget is used to provide a single-line caption for other widgets. It can also contain images.
  • Listbox The Listbox widget is used to provide a list of options to a user.
  • Menubutton The Menubutton widget is used to display menus in your application.
  • Menu The Menu widget is used to provide various commands to a user. These commands are contained inside Menubutton.
  • Message The Message widget is used to display multiline text fields for accepting values from a user.
  • Radiobutton The Radiobutton widget is used to display a number of options as radio buttons. The user can select only one option at a time.
  • Scale The Scale widget is used to provide a slider widget.
  • Scrollbar The Scrollbar widget is used to add scrolling capability to various widgets, such as list boxes.
  • Text The Text widget is used to display text in multiple lines.
  • Toplevel The Toplevel widget is used to provide a separate window container.
  • Spinbox The Spinbox widget is a variant of the standard Tkinter Entry widget, which can be used to select from a fixed number of values.
  • PanedWindow A PanedWindow is a container widget that may contain any number of panes, arranged horizontally or vertically.
  • LabelFrame A labelframe is a simple container widget. Its primary purpose is to act as a spacer or container for complex window layouts.
  • tkMessageBox This module is used to display message boxes in your applications.

Standard attributes

  • Dimensions
  • Colors
  • Fonts
  • Anchors
  • Relief styles
  • Bitmaps
  • Cursors

Geometry Management

  • The pack() Method - This geometry manager organizes widgets in blocks before placing them in the parent widget.
  • The grid() Method - This geometry manager organizes widgets in a table-like structure in the parent widget.
  • The place() Method -This geometry manager organizes widgets by placing them in a specific position in the parent widget.

For further reference visit: tutorialspoint.com/python/python_gui_programming.htm

Enable auto complete in python interpreter

Create a file in your home directory named: .pythonrc
Content of this file:

import rlcompleter, readline
readline.parse_and_bind('tab:complete')

Set the PYTHONSTARTUP variable in your .bashrc or .bash_profile

echo "export PYTHONSTARTUP=~/.pythonrc" >> .bashrc

Reload your .bashrc or .bash_profile

source ~/.bashrc

To test it import a library, write librariname. and hit tab twice

Delete *.pyc

find . -name "*.pyc" -exec rm -rf {} \;

format hex

Use the format() function with a '02x' format.

>>> format(255, '02x')
'ff'
>>> format(2, '02x')
'02'

The 02 part tells format() to use at least 2 digits and to use zeros to pad it to length, x
means lower-case hexadecimal.

The Format Specification Mini Language also gives you X for uppercase hex output, and you can
prefix the field width with # to include a 0x or 0X prefix (depending on wether you used
x or X as the formatter). Just take into account that you need to adjust the field width
to allow for those extra 2 characters:

>>> format(255, '02X')
'FF'
>>> format(255, '#04x')
'0xff'
>>> format(255, '#04X')
'0XFF'

Command completion

$ nano ~/.pythonrc
# ~/.pythonrc
# enable syntax completion
try:
    import readline
except ImportError:
    print("Module readline not available.")
else:
    import rlcompleter
    readline.parse_and_bind("tab: complete")
$ nano ~/.bashrc
export PYTHONSTARTUP=~/.pythonrc

nose debug

nosetests --debug=nose,nose.importer --debug-log=nose_debug <your usual args>

Publish Python Package

Upload a package to pypi. This would make your package available with:

pip install mypackage

With setup.py

nano ~/.pypirc

[distutils]
          index-servers =
            pypi
            pypitest
          
          [pypi]
          repository=https://upload.pypi.org/legacy/
          username=your_username
          password=your_password
          
          [pypitest]
          repository=https://testpypi.python.org/pypi
          username=your_username
          password=your_password

Adjust .pypirc permissions

chmod 600 ~/.pypirc

In your project, create a setup.py

import os
from setuptools import setup, find_packages

# from distutils.core import setup

# allow setup.py to be run from any path
os.chdir(os.path.normpath(os.path.join(os.path.abspath(__file__), os.pardir)))


def here(name):
    return os.path.join(
        os.path.dirname(os.path.abspath(__file__)),
        name)


def read(name, mode='rb', encoding='utf8'):
    os.system('pandoc --from=markdown --to=rst --output=README.rst README.md')
    if os.path.exists('README.rst'):
        long_description = open('README.rst').read()
    else:
        try:
            with open(here(name), mode) as fp:
                long_description = fp.read()
        except IOError:
            return 'Error generating long description: {} File not found'.format(here(name))
    return long_description

# Development Status :: 1 - Planning
# Development Status :: 2 - Pre-Alpha
# Development Status :: 3 - Alpha
# Development Status :: 4 - Beta
# Development Status :: 5 - Production/Stable
# Development Status :: 6 - Mature
# Development Status :: 7 - Inactive


license_classifiers = {
    'MIT license': 'License :: OSI Approved :: MIT License',
    'BSD license': 'License :: OSI Approved :: BSD License',
    'ISC license': 'License :: OSI Approved :: ISC License (ISCL)',
    'Apache Software License 2.0': 'License :: OSI Approved :: Apache Software License',
    'GNU General Public License v3': 'License :: OSI Approved :: GNU General Public License v3 (GPLv3)'
}

setup(
    name='CheKnife',
    version='v0.0.6b2',
    packages=find_packages(),
    url='https://git.herrerosolis.com/Misc/CheKnife',
    download_url='https://git.herrerosolis.com/Misc/CheKnife/-/archive/v0.0.6b2/CheKnife-v0.0.6b2.tar.gz',
    license='MIT license',
    author='Rafael Herrero Solis',
    author_email='rafahsolis@hotmail.com',
    keywords=['CheKnife', 'Swiss', 'Army', 'Knife', 'Swiss Army Knife'],
    description='Python Utilities',
    long_description=read('README.md'),
    test_suite='nose.collector',
    tests_require=['nose', 'six'],
    install_requires=[
        'six>=1.10.0',
        'future>=0.16.0',
        'pycryptodome>=3.6.1',
        'configparser>=3.5.0'
    ],
    classifiers=[
        'License :: OSI Approved :: MIT License',
        'Development Status :: 3 - Alpha',
        'License :: OSI Approved :: Apache Software License',
        'Programming Language :: Python :: 3.6',
    ],
)

nano setup.cfg

[metadata]
          description-file = README.md

nano LICENSE

The MIT License
          
          SPDX short identifier: MIT
          
          Copyright <YEAR> <COPYRIGHT HOLDER>
          
          Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
          
          The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
          
          THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

Test and upload

python setup.py register -r pypitest
python setup.py sdist upload -r pypitest
python setup.py register -r pypi
python setup.py sdist upload -r pypi

With twine

https://packaging.python.org/tutorials/packaging-projects/

Change PyPi

Using PyPi Local Repository

nano ~/.pip/pip.conf

[global]
          index-url= http://10.255.0.21/pypi/simple
          trusted-host= 10.255.0.21

Using pip behind proxy

sudo pip --proxy http://proxy.hell:3128 install requests

Publish to PyPi

Create the file ~/.pypirc

[distutils]
          index-servers =
              pypi
              pypitest
          
          [pypi]
              repository=https://pypi.python.org/pypi
              username=
              password=
          
          [pypitest]
              repository=https://test.pypi.org/legacy
              username=
              password=
          

Test

python setup.py register -r pypitest
python setup.py sdist upload -r pypitest

Upload

python setup.py register -r pypi
python setup.py sdist upload -r pypi

requirements.txt

Platform conditional

psycopg2-binary~=2.9.7; platform_system == "Linux"
psycopg2~=2.9.6; platform_system == "Windows"

AWS boto3 EC2

import boto3
client = boto3.client('ec2', aws_access_key_id="AKI.............Q", aws_secret_access_key="2..........................s", region_name='us-east-1')
ec2_regions = [region['RegionName'] for region in client.describe_regions()['Regions']]
instances = []
for region in ec2_regions:
    conn = boto3.resource('ec2', aws_access_key_id="AKIAJBWPORUPPGNSUWXQ", aws_secret_access_key="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", region_name=region)
    instances += conn.instances.filter()
i = instances[0]
i.public_dns_name
i.public_ip_address