Aide mémoire d'Arthur pour l'utilisation de Python

Version : 1.36.1 4878 (2024-12-18) 20241218130708 ⇑

Auteurs :

Arthur TOROSSIAN

Résumé :

Cette page contient une information concise sur le langage python.

1 Priorité des opérateurs (precedence)

Par oredre croissante :

• lambda

  Lambda expression

• if else
• or
• and
• not x
• in, not in, is, is not, <, <=, >, >=, <>, !=, ==
• |

  Bitwise OR

• ^

  Bitwise XOR

• &

  Bitwise AND

• <<, >>

  Shifts

• +, -
• *, /, //, %
• +x, -x, ~x

  Positive, negative, bitwise NOT

• **

  Exponentiation

• x[index], x[index:index], x(arguments...), x.attribute

  Subscription, slicing, call, attribute reference

• (expressions...), [expressions...], {key: value...}, `expressions...`

  Binding or tuple display, list display, dictionary display, string conversion

docs.python.org

2 Les types

2.1 Les constantes

• PEP 8 – Style Guide for Python Code
• pconst(simon-ritchie)
• named_constants(hmeine)
• typing.Final
• ★Enum Get Value – Five Best Methods
• ★★★ enum

class indCont(IntEnum):
    @classmethod
    def nm( name, defVal=None):        
        return getattr( name , defVal )

class gbI(indCont):
    dH             = 0  # gbI.dH deltaHeight
    dT             = 1  # gbI.dT
    dTcMax         = 2  # gbI.dTcMax
    dWaterFlowMax  = 3  # gbI.dWaterFlowMax
    dl             = 4  # gbI.dl
    upLevel        = 5  # gbI.upLevel
    downLevel      = 6  # gbI.downLevel
    wettedLength   = 7  # gbI.wettedLength
    dryLength      = 8  # gbI.dryLength
    emergedLength  = 9  # gbI.emergedLength  -> flt.emergedLength
    immergedLength = 10 # gbI.immergedLength -> flt.immergedLength
    filterFlow     = 11 # gbI.filterFlow -> flt.waterFlow
    imFlow         = 12 # gbI.imFlow -> flt.imFlow
    emFlow         = 13 # gbI.emFlow -> flt.emergedFlow
    wetTc          = 14 # gbI.wetTc -> flt.wettedTc

if __name__ == '__main__':
    #---------------------------------------------------------------------------
    print( gbI )
    print( list(gbI) )
    print( "dT", gbI( gbI.dT) )
    print( "downLevel", getattr( gbI, "downLevel" , None ))
    print( "immergedLength", gbI.nm( "immergedLength") )
    print( gbI.dH.name,  gbI.dH )
    print( gbI.dT.name, gbI.dT )
    try :
        gbI.dH = 4
    except AttributeError as err :
        print(err)

2.2 Les dates

2.2.1 git/pythonTool/pattern.py

#-------------------------------------------------------------------------------
import time # https://docs.python.org/fr/3/library/time.html#time.gmtime
# L'epoch est le point de départ du temps, le résultat de time.gmtime(0) est 
# le 1er janvier 1970 à 00:00:00 
#-------------------------------------------------------------------------------
def getTimeAsFloat() :
    return time.time() # absolue time in seconds
#-------------------------------------------------------------------------------
def getTimeAsStr( fmt ):
    return time.strftime("%Y/%m/%d_%H:%M:%S",fmt)
#-------------------------------------------------------------------------------
timeSec = getTimeAsFloat()              # time in seconds 
gmtTime = time.gmtime( timeSec )        # gmt Greenwich Mean Time 
                                        # (Temps moyen de Greenwich)
locTime = time.localtime( timeSec )     # local time
#-------------------------------------------------------------------------------
print( "(s) time", timeSec ) 
print( "gmt time", getTimeAsStr( gmtTime ) ) 
print( "loc time", getTimeAsStr( locTime ) )

(s) time 1734213182.5401628
gmt time 2024/12/14_21:53:02
loc time 2024/12/14_22:53:02

2.2.2 get_datetime_str

def get_datetime_str():
    return time.strftime("%Y-%m-%d_%H-%M-%S",time.localtime())

def get_date_year( date ):
    return time.strftime("%Y",date)

def get_date_month( date ):
    return time.strftime("%m",date)

def get_date_day( date ):
    return time.strftime("%d",date)

2.2.3 date_str2date_float

def date_str2float( sdate ):
    ds = time.strptime( sdate,"%d/%m/%Y"  )
    d = time.mktime( ds )
    return d

2.2.4 date_float2date_str

def date_float2date_str( d ):
    ds = time.strftime("%d/%m/%Y", time.gmtime(d) )
    return ds

2.2.5 date_float_add_days

def date_float_add_days( d, nb ):
    nd = d + 3600*24*nb
    return nd

2.2.6 time2str str2time getAcTimeOfFile timeToFile fileToTime

timeForat = "%Y-%m-%d_%H-%M-%S"
def time2str( t=None ):
    st = time.localtime( t )
    return time.strftime( timeForat , st )
def str2time( text ) :
    st = time.strptime( dT, timeForat )
    return time.mktime( st )
def getAcTimeOfFile( filename ):
    infoStat = os.stat( filename )
    st = time.localtime( infoStat.st_atime )
    t = time.mktime( st )
    return t
def timeToFile( filename, t ):
    dT = time2str( t )
    tofile( filename, dT )
def fileToTime( filename ):
    dt = fromfile( filename )

3 Système et codage das chaînes de caractères

Parfois le codage avec le système a des problèmes :

print(sys.getdefaultencoding())
print(sys.stdout.encoding)

#!/usr/bin/python
# -*- coding: UTF-8 -*-

print( "OS encoding : ", locale.getpreferredencoding())

Pour python3 :

def printRaw( *text ):
    myout = open(1, "w", encoding = "UTF-8", closefd = False)
    print(*text, file=myout)
    myout.flush()
    myout.close()

4 Programme

4.1 Versionnement

4.1.1 Obtenir la signature numérique (hashage sha) du versionnement par git

4.1.1.1 subprocess.check_output - git describe or git show

def getVersionningSignature(  ):
    try : 
        # label = "-"+subprocess.check_output(["git", "describe"]).strip(
        # ).decode('utf-8')
        sign = "_sha_"+subprocess.check_output(
                            ["git", "show","-s","--format=%h"]).decode('utf-8')    
    except FileNotFoundError as err :
        sign = "".encode('utf-8')
    return sign.strip()

4.1.1.2 subprocess.check_output - git rev-parse HEAD

import subprocess
def get_git_revision_hash() -> str:
    return subprocess.check_output(['git', 'rev-parse', 'HEAD']).decode('utf-8').strip()

def get_git_revision_short_hash() -> str:
    return subprocess.check_output(['git', 'rev-parse', '--short', 'HEAD']).decode('utf-8').strip()

4.1.1.3 gitpython

pip install gitpython

import git
repo = git.Repo(search_parent_directories=True)
sha = repo.head.object.hexsha

4.1.2 Application

################################################################################
# release.V1.V2.V3.V4
# release x(n), rpa, ra, rb, rc(n), rl :
#         x(n) : x1 x2 , x3 ... x400 ... (in development)
#         rpa  : Release Pre-Alpha for unit or intergation tests
#         ra   : release alpha final tests in development team "recette usine"
#         rb   : release beta test in client test team "recettes métier"
#         rc(n): release candidate 1, 2, 3, ... test in client test team
#         rl   : release live "live release" or gd (gold) or pr (prod)
# 
#         V1 : architecture modification
#         V2 : function modifications
#         V3 : implementation of functions
#         V4 : bug corrections 
VERSION = "x1-0.0.0.0"
################################################################################
APP_PATH, APP_NAME = os.path.split(__file__)

4.1.3 Library

#!/usr/bin/python
# -*- coding: UTF-8 -*-
################################################################################
# API identification file with list of interface published and accessible items.
################################################################################
# module identification file with api sub module.
#
# [release-]current.revision.age
#
# [release-]apiVersion.revisionOfThisVersion.ageOfThisApi
#
# exemple : 3.1.2
#             if you use API 1.x.x you can use it : 3-2=1 
# current :
#         interface, API version
# revision:
#         implementation of API version
# age     :
#         ascendant compatibility between API
#         versions compatible width API : current, current-1, ...(current - age) 
#
# release x(n), rpa, ra, rb, rc(n), rl :
#         x(n) : x1 x2 , x3 ... x400 ... (in development)
#         rpa  : Release Pre-Alpha for unit or intergation tests
#         ra   : release alpha final tests in development team "recette usine"
#         rb   : release beta test in client test team "recettes métier"
#         rc(n): release candidate 1, 2, 3, ... test in client test team
#         rl   : release live "live release" or gd (gold) or pr (prod)
VERSION = "x1-0.0.0"
ABSTRACT_MSG="call of abstract method"

4.2 En-tête

#!/usr/bin/python
# -*- coding: UTF-8 -*-

4.3 main

if __name__ == '__main__' :    
    pass

4.4 minmal

#!/usr/bin/python
# -*- coding: UTF-8 -*-
import os, sys, re

if __name__ == '__main__' :    
    pass

4.5 Exception, raise, try

Lire d'abord La bonne façon de gérer les exceptions

4.5.1 solution élégante pour tracer et informer

Exemple d'utilisation

def debugInfo( pm ):
        global debugIndex
        traceback.format_stack()
        # frame,filename, num, func = inspect.getouterframes(
        #                                     inspect.currentframe())[1][:4]
        debugIndex += 1
        print("")
        print("id(pm)", id(pm) )
        print("debugIndex",debugIndex)
        for ll in traceback.format_stack()[:-1] :
            print( "    "+ll )
        print(pm)
        print("%s" %(pm.onGetInfo))
        print("%s %s" %(pm.pressureDropMaxTime, pm.pressureDropMax ))
        print("Enter");input()

import traceback
import sys
from testLib import exceptionProcess
                
def myFunction2( a ):
    myFunction( a )

class MyClass :
    def __init__(self,a):
        self.a = a
    def do(self):
        try :
            a = self.a / 0             
        except Exception as err :
            errMsg = exceptionProcess(err, sys.exc_info(), 
                                                    traceback.format_stack() )
            print(errMsg)
            exit(1)
        
def myFunction( a ):
    cl = MyClass(a)
    cl.do()
    print("End of myFunction")        
        
if __name__ == '__main__' :
    pass
    myFunction2( 3 )

#!/usr/bin/python
# -*- coding: UTF-8 -*-
import inspect
import traceback
################################################################################
def exceptionProcess(err, exeInfo, lastStack ):
    """
import sys
import traceback
import inspect
    
use :
errMsg = exceptionProcess(err, sys.exc_info(), traceback.format_stack() )
"""    
    frame,filename, num, func = inspect.getouterframes(
                                            inspect.currentframe())[1][:4]    
    last =  traceback.format_list(traceback.extract_tb(exeInfo[2]))[0]
    history = lastStack
    txt = ""
    for x in history[:-1] :
        txt += x
    txt += last
    txt += "catched p:%s\nline:%s func:%s\n" % (filename, num, func)    
    txt += "ERROR " + str(err)    
    return txt
################################################################################

4.5.2 Obtenir nom de fichier, numéro de ligne et nom de fonction

4.5.2.1 Cas simple mais non utile

import inspect

raisehead = "Error in %s:%s:%s\n" %inspect.getframeinfo(inspect.currentframe())[:3]
raise xmlBaseError("%sNo valid cardinality=%s for node named grammar in grammar" %(raisehead,node_nb))

4.5.2.2 Obtenir nom de fichier, numéro de ligne et nom de fonction de l'appelant

class xmlBaseError( Exception ) :
    def __init__(self, value):
        frame,filename, num, func = inspect.getouterframes(inspect.currentframe())[1][:4]
        path, name = os.path.split(filename)
        head="%s:%s:%s\n    " % ( name, num, func)
        self.value = head+value
    def __str__(self):
        return self.value

4.5.3 Un exemple simple pour tout exceptions en affichant la trace

#!/usr/bin/env python

try :
    raise ...

except SomeException as err:
    errtrbk = traceback.format_exc()
    print "ERROR:\n%s\%s" %(str(err),errtrbk))

    print u"MyException raised", e.__class__.__name__, e
else :
    print u"Unkown exeption :"

4.5.4 Un exemple avec importation

exception_test.py  :

class MyException( Exception ) :
    pass

fonctions.py :

from exception_test import MyException

def f() :
    raise MyException(u"test")

raise.py :

#!/usr/bin/env python

from exception_test import MyException
from fonctions import f

try :
    f()
except MyException,e :
    print u"MyException raised", e.__class__.__name__, e
else :
    print u"Unkown exeption"

Résultats :

MyException raised MyException test

4.6 Options en ligne de commande

#!/usr/bin/python
# -*- coding: ISO8859-1 -*-

import os,sys, getopt

################################################################################
# Help
################################################################################
def help() :
    print u"""\
Help on %(app)s :
-------------------    
%(app)s -x -x AAA --XXXX --XXXX AAAA  
    -h :
        print this help
    """ % ({"app":__file__})   
################################################################################
# Main
################################################################################

if __name__ == '__main__' :
    short = u"xh"
    long = []
    opts,args=getopt.getopt( sys.argv[1:],short, long)
    options = list()
    for opt,val in opts:
        options.append( opt )
    
    for opt,val in opts:
        #if opt==u'-x' :
        #    ...        
        #if opt==u'--XXXXXX' :
        #    ...   
        pass

    if (len(args)==0) and (len(opts)==0) :
        options.append( u"-h" )

    if u'-h' in options  :
        help()
    else :
        # lauch requests ...
        pass

googWay

4.7 Options en ligne de commande : getopt, help, CParams

generated by gen_main_options.py :

#!/usr/bin/python
# -*- coding: UTF-8 -*-
#***********************************************************
"""\
TODO
"""
import os
import sys
import getopt
#import string
#import re
#import time
#import shutil
################################################################################
# Version
################################################################################
APP_PATH, APP_NAME = os.path.split(__file__)
VERSION = u"0.1.0"
################################################################################
# Exceptions
################################################################################
class CAppException(Exception):
    """\
the exception class of the module
    """
    pass
################################################################################
# The process
################################################################################
def do_someThing(params):
    """\
the main process
    """
    print params.longParamVal
    raise CAppException("not good for us")
################################################################################
# params
################################################################################
class CParam(object):
    """\
parameters of the process
    """
    def __init__(self):
        """\
initialize all default values of parameters
        """
        self.errorMsg = ""
        self.shortParamBool = None
        self.shortParamVal = None
        self.longParamBool = None
        self.longParamVal = None

    def write_errorOnStdOutput(self):
        """\
print the parameter errors
        """
        for ms in self.errorMsg.split('\n'):
            if ms.strip() != "":
                print "ERROR: %s" % ms
        exit(1)

    def checkParam(self):
        """\
check parameters
        """
        ok = True
        self.errorMsg = ""
        if self.shortParamBool != None:
            ok = True
            self.errorMsg += " ... \n"

        if self.shortParamVal != None:
            ok = True
            self.errorMsg += " ... \n"

        if self.longParamBool != None:
            ok = True
            self.errorMsg += " ... \n"

        if self.longParamVal != None:
            ok = True
            self.errorMsg += " ... \n"
        if not ok:
            self.write_errorOnStdOutput()
        return ok

################################################################################
# Help
################################################################################
def write_helpOnStdOutput():
    u"""\
print the help of the application
    """
    print u"""
Help on %(app)s:
-------------------
* %(app)s version %(version)s
    %(app)s -h

* Options:
    -h:
      Print this help.
    -b
      ...
    -v <val>
      ...
    --longBool
      ...
    --longVal <val>
      ...""" % ({u"app": APP_NAME, u"version": VERSION})

################################################################################
# Main
################################################################################
def executeMainProcess():
    """\
    execute main process
    """
    try:
        appParam = CParam()
        shortOption = u"hsv:"
        longOption = [u"longBool", u"longVal="]
        dohelp = False

        opts = getopt.getopt(sys.argv[1:], shortOption, longOption)[0]

        for opt, val in opts:

            if opt == u'-h':
                dohelp = True
                write_helpOnStdOutput()

            if opt == u'-s':
                appParam.shortParamBool = True

            if opt == u'-v':
                appParam.shortParamVal = val

            if opt == u'--longBool':
                appParam.longParamBool = True

            if opt == u'--longVal':
                appParam.longParamVal = val

            else:
                pass

        appOk = appParam.checkParam()
        if (not dohelp) and appOk:
            do_someThing(appParam)

    except Exception as ex:
        info = sys.exc_info()
        raise CAppException("\nERRORS %s" % str(ex)), None, info[2]

if __name__ == u'__main__':
    executeMainProcess()

4.8 execute - command - popen

Al1	Utiliser plutôt subrocess (os.pepen est obsolète)

def execute( command  ) :
    print "cmd : ", command
    e = os.popen( command + " 2>&1" , "r" )
    out = e.read()    
    rc =  e.close()
    if rc == None :
        rc= 0
    std_out = out
    std_err="See Standard out"
    return rc, std_out, std_err

En mode pas à pas; interception de la sortie standard en cours

#!/usr/bin/python
# -*- coding: UTF-8 -*-
import re
import subprocess
import sys
pat = re.compile("(\d\/\d)")
def do(exe):
    sys.stdout.flush()
    try:
        p = subprocess.Popen(exe, stdout = subprocess.PIPE, stderr = \
                                                subprocess.STDOUT, shell = True)
    except Exception as err:
        print err
        sys.stdout.flush()
    encore = True
    while(encore):
        retcode = p.poll()  # returns None while subprocess is running
        sys.stdout.flush()
        line = p.stdout.readline()
        val = pat.findall(line)
        if len(val) > 0:
            print val[0]
        sys.stdout.flush()
        encore = retcode == None
    print "end"
if __name__ == '__main__':
    print "start"
    do("python cmd.py")
    print "end"

le programme cmd.py

#!/usr/bin/python
# -*- coding: UTF-8 -*-
import sys
import time
nb = 10
for x in range(nb):
    print "toto %d/%d" % (x, nb)
    sys.stdout.flush()
    time.sleep(1.0)

4.9 execute - command - subprocess - popen

def executeInShell( shellcmd  ) :
    process = subprocess.Popen( shellcmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE )    
    stdoutdata, stderrdata = process.communicate()        
    return process.returncode, stdoutdata, stderrdata

Exécution en mode synchrone avec capture progressive de

#!/usr/bin/python
# -*- coding: UTF-8 -*-
import re
import subprocess
import sys
# patTelemacIteration = re.compile("(\d+\/\d+)")
patTelemacIteration = re.compile("ITERATION +(\d+) +TEMPS")

def executeExternalShellCommand(exe):
    sys.stdout.flush()
    try:
        p = subprocess.Popen(exe, stdout = subprocess.PIPE, stderr = \
                                                subprocess.STDOUT, shell = True)
    except Exception as err:
        print err
        sys.stdout.flush()
    encore = True
    while(encore):
        retcode = p.poll()  # returns None while subprocess is running
        sys.stdout.flush()
        line = p.stdout.readline()
        print line
        val = patTelemacIteration.findall(line)
        print val
        if len(val) > 0:
            print int(val[0])
        sys.stdout.flush()
        encore = retcode == None
    print "end"
if __name__ == '__main__':
    print "start"
    executeExternalShellCommand("cd ...; python cmd.py")
    print "end"

#!/usr/bin/python
# -*- coding: UTF-8 -*-
import sys
import time
nb = 10
for x in range(nb):
    print " ITERATION      %d  TEMPS : 15 MN   0.0000 S   (      900.0000 S) IT\
ERATION      200  TEMPS" % (x)
    sys.stdout.flush()
    time.sleep(1.0)

4.9.1 Get directly the output

result = subprocess.check_output("gsettings get org.gnome.desktop.background picture-uri", shell=True)

4.10 isLanched

def isLanched( procName ):
    import psutil, string
    pid = os.getpid()
    patText = re.escape(procName)
    pat = re.compile( ".*%s.*" % patText )
    ok = False
    for proc in psutil.process_iter():
        if pid != proc.pid :
            if pat.match( string.join(proc.cmdline()," ") )  :
                ok=True
    return ok

4.11 Ajouter des répertoire dans le path python en dynamique

import sys
sys.path.append("/home/me/mypy")

5 La commande print

5.1 Afficher sur la sortie d'erreurs (stderr)

print >> sys.stderr, "Error ..."

en Python 3 :

print(5, "toto", file=sys.stderr)

Sa propre fonction:

def conErrOut(*args):
    print(*args, file=sys.stderr, **kwargs)

6 Design pattern, POO, Algorithmes

6.1 POO Programmation orientée objet

6.1.1 class attribute - instance attribute

class CObj(object):
  classAttr = 0
  def __init__(self, name, instance_attr):
    self.name = name
    self.objAttr = instance_attr
  def __str__(self):
    return "%s %s %s %s" %(self.name, self.classAttr, self.objAttr, self.__dict__)

if __name__ == '__main__':
    
    obj1 = CObj("A", 1)
    obj2 = CObj("B", 2)

    
    print (CObj.classAttr)
    print (obj1.classAttr)
    print (obj2.classAttr)
    print( obj1 )
    print( obj2 )
    print()
    
    CObj.classAttr += 1
    print (CObj.classAttr)
    print (obj1.classAttr)
    print (obj2.classAttr)    
    print( obj1 )
    print( obj2 )
    print()

    obj1.classAttr += 1    # ATTENTION PIEGE
    print (CObj.classAttr)
    print (obj1.classAttr)
    print (obj2.classAttr)    
    print( obj1 )
    print( obj2 )
    print()

    obj2.classAttr += 1    # ATTENTION PIEGE
    print (CObj.classAttr)
    print (obj1.classAttr)
    print (obj2.classAttr)    
    print( obj1 )
    print( obj2 )
    print()
	
    CObj.classAttr += 1
    print (CObj.classAttr)	
    print (obj1.classAttr)
    print (obj2.classAttr)    
    print( obj1 )
    print( obj2 )
    print()

    print (CObj.objAttr)   # ERROR

0
0
0
A 0 1 {'objAttr': 1, 'name': 'A'}
B 0 2 {'objAttr': 2, 'name': 'B'}

1
1
1
A 1 1 {'objAttr': 1, 'name': 'A'}
B 1 2 {'objAttr': 2, 'name': 'B'}

1
2
1
A 2 1 {'objAttr': 1, 'name': 'A', 'classAttr': 2}
B 1 2 {'objAttr': 2, 'name': 'B'}

1
2
2
A 2 1 {'objAttr': 1, 'name': 'A', 'classAttr': 2}
B 2 2 {'objAttr': 2, 'name': 'B', 'classAttr': 2}

2
2
2
A 2 1 {'objAttr': 1, 'name': 'A', 'classAttr': 2}
B 2 2 {'objAttr': 2, 'name': 'B', 'classAttr': 2}

Traceback (most recent call last):
  File "testP.py", line 55, in <module>
    print (CObj.objAttr)
AttributeError: type object 'CObj' has no attribute 'objAttr'

6.2 Sérialisation

6.2.1 pickle

################################################################################
import pickle
def objToFile(  obj, fileName  ):
    with open(fileName, 'wb') as handle:
        pickle.dump( obj, handle, protocol=pickle.HIGHEST_PROTOCOL)
#-------------------------------------------------------------------------------
def objFormFile(  fileName  ):
    with open(fileName, 'rb') as handle:
        obj = pickle.load(handle)
    return obj
################################################################################

6.3 POO

6.3.1 CDictList

class CDictList :
    def __init__( self, name ) :
        self.name = name
        self.d = dict()
        self.l = list()
        
    def add( self, obj, name  ) :
        if name in self.d :
            raise Exception( 
                    "object named '%s' is already in CDictList named '%s'" 
                    % ( name, self.name ) )
        self.d[ name ] = obj
        self.l.append( obj )

6.4 Génération de code

6.4.1 Générer une fonction

correctionCoefficient=[ (-5000,-5000), (-4000,-4800), (-1000,-3000), (-50,-1000), (0,0), (400, 5), (500, 10), (1000, 1000),(5000, 5000) ]
def createCorrectionFunction(  ) :
    fileName="func.py"
    txt = "#!/usr/bin/python\n"
    txt += "# -*- coding: UTF-8 -*-\n"    
    txt += "def hcorection( val ) :\n"
    txt += "    nval=val\n"
    
    a=correctionCoefficient[0]
    for b in correctionCoefficient[1:] :     
        txt += "    if (val > %d) and (val <=%d) :\n" %( a[0], b[0] )
        txt += "        nval =  ( val- %f )*%f +  %f\n" %( a[0], float(b[1]-a[1])/(b[0]-a[0])  ,a[1] )
        a=b
    txt += "    return nval\n"
    tofile( fileName, txt )

Ce qui donne la fonction suivante :

Fichier:Create function ex01.png

Fig. n°1 Exemple

6.5 Design pattern

6.5.1 Singleton

Voici un petit code écrit en python qui explique comment écrire un singleton en langage python. Notez bien que dans la méthode __init__ il faut faire quelques vérifications notamment avec les listes.

class CSingleton( object ):
    class_instance = None
    def __init__( self, a ):
        print "CSingleton init"
        self.vitesse = a
        if not hasattr( self, "alist") :
            self.alist = list()
    
    def __new__( typ, *args, **kwargs ):
        print "CSingleton new", typ.class_instance        
        if typ.class_instance == None :
            obj = object.__new__( typ, *args, **kwargs)
            typ.class_instance = obj
        else :
            obj = typ.class_instance        
        return obj

    def addObject(self, obj):
        self.alist.append( obj )

if __name__ == '__main__' :
    a = CSingleton( 1 )
    a.addObject( "Lundi" )
    print "a.vitesse=",a.vitesse
    print "a.alist=",a.alist

    b = CSingleton( 2 )
    b.addObject( "Mardi" )
    print "b.vitesse=",b.vitesse
    print "b.alist=",b.alist

    c = CSingleton( 3 )
    c.addObject( "Mercredi" )
    print "c.vitesse=",c.vitesse
    print "c.alist=",c.alist

    print "a.vitesse=",a.vitesse
    print "a.alist=",a.alist
    print "b.vitesse=",b.vitesse
    print "b.alist=",b.alist
    print "c.vitesse=",c.vitesse
    print "c.alist=",c.alist
    print "id(a)=",id(a)
    print "id(b)=",id(b)
    print "id(c)=",id(c)

Ce qui donne :

python singleton.py
CSingleton new None
CSingleton init
a.vitesse= 1
a.alist= ['Lundi']
CSingleton new <__main__.CSingleton object at 0xb7cf8ecc>
CSingleton init
b.vitesse= 2
b.alist= ['Lundi', 'Mardi']
CSingleton new <__main__.CSingleton object at 0xb7cf8ecc>
CSingleton init
c.vitesse= 3
c.alist= ['Lundi', 'Mardi', 'Mercredi']
a.vitesse= 3
a.alist= ['Lundi', 'Mardi', 'Mercredi']
b.vitesse= 3
b.alist= ['Lundi', 'Mardi', 'Mercredi']
c.vitesse= 3
c.alist= ['Lundi', 'Mardi', 'Mercredi']
id(a)= 3083833036
id(b)= 3083833036
id(c)= 3083833036

6.6 Algorithmes

6.6.1 resgression linéaire en 3 dimensions pour obtenir un plan optimum pour une nuage de points

import numpy as np
################################################################################
nbpts = 10 # nombre de points par axe
a = 5. # paramètres du plan
b = 3.
c = 2.
d = 10.
sigma = 0.1 # écart type
################################################################################
def planXyz(x, y, z,  A):
    return A[0]*x + A[1]*y + A[2]*z + A[3]
################################################################################
def lienarRegressionXyz(X,Y,Z):
    points = np.hstack(
            (
                X, Y, Z, np.ones_like(X) 
            ) 
        )
    resultat = np.linalg.lstsq(points, T)
    aopt, bopt, copt, dopt = resultat[0]
    erreur = resultat[1][0]/(nbpts*nbpts)
    return  aopt, bopt, copt, dopt,erreur 
################################################################################
x = np.linspace(0, 1, nbpts)
y = np.linspace(0, 1, nbpts)
z = np.linspace(0, 1, nbpts)

grilleX, grilleY, grilleZ = np.meshgrid(x, y, z)
################################################################################
X = grilleX.flatten().reshape(nbpts*nbpts*nbpts, 1)
Y = grilleY.flatten().reshape(nbpts*nbpts*nbpts, 1)
Z = grilleZ.flatten().reshape(nbpts*nbpts*nbpts, 1)
T = planXyz(X, Y, Z,  (a, b, c, d)) + sigma*np.random.randn(nbpts*nbpts*nbpts, 
                                                                            1)
################################################################################
a,b,c,d,e = lienarRegressionXyz(X,Y,Z)
print( 
    "a=%f b=%f c=%f d=%f χ²=%f" %( a,b,c,d,e ))

6.6.2 Filtre passe bas - exemple

from scipy import signal
import matplotlib.pyplot as plt
import numpy as np

t = np.linspace(0, 1, 1000, False)  # 1 second
sig = np.sin(2*np.pi*10*t) + np.sin(2*np.pi*20*t)+1.5

class lpFilter:
    def __init__(self, fc, fe):
        self.fc = fc
        self.fe = fe
        self.τ = 1/fc
        self.Te = 1 / fe
        self.α = self.τ / (self.τ + self.Te)
    
    def filter(self, sig):
        yf_0 = sig[0] 
        yf_1 = yf_0
        yf = list()
        for y in sig:
            yf_0 = self.α * yf_1 + (1-self.α)* y 
            yf_1 = yf_0
            yf.append(yf_0)
        return yf

flp = lpFilter( 15, 1000 )
yf = flp.filter( sig )

plt.plot(t, yf, label='yf')
plt.plot(t, sig, label='sig')
plt.legend()
plt.show()

Fichier:FiltrePasseBasLineaireExample.png

Fig. n°2 Exemple filttre linéaire

Utilisation de ellip :

from scipy import signal
import matplotlib.pyplot as plt
import numpy as np

t = np.linspace(0, 1, 1000, False)  # 1 second
sig = np.sin(2*np.pi*10*t) + np.sin(2*np.pi*20*t)+1.5

sos = signal.ellip(8, 1, 100, 5, 'lowpass', fs=1000, output='sos')
filtered = signal.sosfilt(sos, sig)

plt.plot(t, filtered)
plt.plot(t, sig)
plt.show()

Fichier:FiltrePasseBasEllipExample.png

Fig. n°3 ellip exemple

Diagramme des fréquences:

from scipy import signal
import matplotlib.pyplot as plt
import numpy as np

b, a = signal.ellip(4, 5, 40, 100, 'low', analog=True)
w, h = signal.freqs(b, a)
 
plt.semilogx(w, 20 * np.log10(abs(h)))
 
plt.title('Elliptic filter frequency response (rp=5, rs=40)')
plt.xlabel('Frequency [radians / second]')
plt.ylabel('Amplitude [dB]')
plt.margins(0, 0.1)
plt.grid(which='both', axis='both')
plt.axvline(100, color='green') # cutoff frequency
plt.axhline(-40, color='green') # rs
plt.axhline(-5, color='green') # rp
plt.show()

Fichier:FiltrePasseBasDiagrammeDesFrequences.png

Fig. n°4 Diagramme des fréquences - ellip exemple

6.6.3 Filtre passe haut - exemple

Utilisation de ellip :

from scipy import signal
import matplotlib.pyplot as plt
import numpy as np

t = np.linspace(0, 1, 1000, False)  # 1 second
sig = np.sin(2*np.pi*10*t) + np.sin(2*np.pi*20*t)+1.5

fig, (ax1, ax2) = plt.subplots(2, 1, sharex=True)
ax1.set_title('10 Hz and 20 Hz sinusoids')
ax1.axis([0, 1, -2, 3.5])
 
ax2.set_title('After 17 Hz high-pass filter')
ax2.axis([0, 1, -2, 2])
ax2.set_xlabel('Time [seconds]')
 
sos = signal.ellip(8, 1, 100, 17, 'hp', fs=1000, output='sos')
filtered = signal.sosfilt(sos, sig)
 
ax1.plot(t, sig)
ax2.plot(t, filtered)
plt.tight_layout()
plt.show()

6.6.4 Fonction rapide d'interpolation linéaire - Functor

algorithmique

6.6.5 Adapter Rectangle Dans Zone

• Version en Pascal

def AdapterRectangleDansZone( zone_x,zone_y,zone_l,zone_h, rect_l, rect_h, rlimte ) :
  if (zone_h > 0 ) and (rect_h>0) :
      r1 = zone_l / zone_h
      r2 = rect_l / rect_h
      #print r1,r2
      if r1 >= r2 :
          if rlimte > 0 :
              r =  zone_h / rect_h
              if r > rlimte :
                  r = rlimte
                  hh = round( rect_h*r )
                  ll = round( rect_l*r )
                  newrect_y = zone_y  +( zone_h-hh ) / 2
                  newrect_h = hh
                  newrect_x = zone_x  + ( zone_l-ll ) / 2
                  newrect_l = ll
              else :
                  hh = zone_h
                  ll = round( zone_h*r2 )
                  newrect_y = zone_y
                  newrect_h = hh
                  newrect_x = zone_x  + ( zone_l-ll ) / 2
                  newrect_l = ll
          else :
            hh = zone_h
            ll = round( zone_h*r2 )
            newrect_y = zone_y
            newrect_h = hh
            newrect_x = zone_x  + (zone_l-ll) / 2
            newrect_l = ll
      else :
          if rlimte > 0 :
              r =  zone_l / rect_l
              if r > rlimte :
                  r = rlimte
                  hh = round( rect_h*r )
                  ll = round( rect_l*r )
                  newrect_y = zone_y  +( zone_h-hh ) / 2
                  newrect_h = hh
                  newrect_x = zone_x  + ( zone_l-ll ) / 2
                  newrect_l = ll
              else :
                  ll = zone_l
                  hh = round( ll/r2 )
                  newrect_x = zone_x
                  newrect_l = ll
                  newrect_y = zone_y  + ( zone_h-hh ) / 2
                  newrect_h = hh
          else :
              ll = zone_l
              hh = round( ll/r2 )
              newrect_x = zone_x
              newrect_l = ll
              newrect_y = zone_y  + ( zone_h-hh ) / 2
              newrect_h = hh
  else :
      newrect_x = zone_x
      newrect_l = 0
      newrect_y = zone_y
      newrect_h = 0

  return newrect_x,newrect_y,newrect_l,newrect_h

6.6.6 Fonction de comparaison pour les tri : sort

6.6.6.1 Avec python 2

La comparaison suivante range dans l'ordre des valeurs croissantes:

def myCmp(pntA, pntB):
    rc = 0
    if pntA < pntB:
        rc = -1
    elif pntA > pntB:
        rc = 1
    return rc

vals = [ 1, 5, 6, 0, -15 ]
vals.sort(myCmp)
print vals

ce qui donne :

[-15, 0, 1, 5, 6]

6.6.6.2 Avec Python 3

Le tri se fait dans l'ordre croissant !

Pour les tuples ou listes imbriqués, tri sur l'indice n

myList = sorted(myList, key = lambda els: els[n])

sprocketTeethPointList1.sort( key = lambda pnt: pnt.angle)

ou

sprocketTeethPointList1 = sorted(sprocketTeethPointList1, 
                             key = lambda pnt: pnt.angle)

ou

def angledPointKey(key ):
    return key.angle

sprocketTeethPointList1 = sorted(sprocketTeethPointList1, key = angledPointKey )

ou

def cmpIntAPy3( aa ):
    a = aa.serviceDate
    if a == None :
        a = aa.prototypeDate    
    return a

self.intAs = sorted(self.intAs, key = cmpIntAPy3 )

6.6.6.3 Avec Python 3 à la manière de Python 2

def angledPointClass( myCmpFunc ) :
    class K :
        def __init__(self, obj, *args):
            self.obj = obj
        def __lt__(self, other):
            return myCmpFunc(self.obj, other.obj) < 0
        def __gt__(self, other):
            return myCmpFunc(self.obj, other.obj) > 0
        def __eq__(self, other):
            return myCmpFunc(self.obj, other.obj) == 0
        def __le__(self, other):
            return myCmpFunc(self.obj, other.obj) <= 0
        def __ge__(self, other):
            return myCmpFunc(self.obj, other.obj) >= 0
        def __ne__(self, other):
            return myCmpFunc(self.obj, other.obj) != 0
    return K        

def angledPointCmp(pntA, pntB):
    rc = 0
    pntA = pntA.angle 
    pntB = pntB.angle
    if pntA < pntB:
        rc = -1
    elif pntA > pntB:
        rc = 1
    return rc   

sprocketTeethPointList1.sort( key=angledPointClass(angledPointCmp) )

7 Clavier, Input device

7.1 get_char

voir curses.window.getch() pour une implémentation sous linux.

if os.name=="nt" :
    import msvcrt
    def get_char() :
        if msvcrt.kbhit() :
            return msvcrt.getch()
        else :
            return None
else :
    print "Error : No get_char()"
    exit(1)

7.2 tell me Yes or No

def tellmeYesNo( msg ):
    encore = 1
    choices=['y', 'n', 'yes', 'no']
    schoices='/'.join( choices )
    rep=None
    while encore :
        rep=raw_input( msg+'(%s): ' % schoices ).lower()
        encore = not rep in choices
        if encore :
            print "Choices are : '%s'" %( schoices )
    return rep

8 Interface graphique

8.2 Sous Windows sans console ou terminal

• voir noTerminal

9 Exception et Erreurs

9.1 La bonne façon de gérer les exception pour un débogage

#!/usr/bin/python
# -*- coding: UTF-8 -*-
import re, os, sys
class CMyException( Exception ) :
    pass
def raiseCMyException( e, msg ) :
    raise CMyException("\nMyInfo\n"+str(e)).with_traceback(sys.exc_info()[2])
def f1() :    
        a = int( "1" )
        b = int( "1x" )                
def f2():
    try :
        f1()
    except Exception as e :
        # old raise CMyException("\nMyInfo\n"+str(e)), None, info[2]             
        raise CMyException("\nMyInfo\n"+str(e)).with_traceback(sys.exc_info()[2])
def f3():
    f2()            
def f4():
    f3()        
try :
    f4()
except Exception as e :
    raise CMyException("\nMyInfo in main\n"+str(e)).with_traceback(sys.exc_info()[2])

Ce qui donne :

Traceback (most recent call last):
  File "exception.py", line 23, in <module>
    f4()
  File "exception.py", line 21, in f4
    f3()        
  File "exception.py", line 19, in f3
    f2()            
  File "exception.py", line 14, in f2
    f1()
  File "exception.py", line 11, in f1
    b = int( "1x" )                
__main__.CMyException: 
MyInfo in main

MyInfo
invalid literal for int() with base 10: '1x'

9.2 error_in_function

def error_in_function( msg, niv=1 ):
    fonction_name  = sys._getframe(niv).f_code.co_name
    raise Exception( "Error : %s : %s" %( fonction_name, msg )  )

9.3 param_isnot_set

def param_isnot_set( param, param_name ):
    if param == None :
        error_in_function( "%s is not set" % param_name, niv=2 )

9.4 param_isnot_file

def param_isnot_file( pathfilename ):
    if not os.path.isfile( pathfilename ) :
        error_in_function( "'%s' is not a file" % pathfilename, niv=2 )

9.5 param_isnot_dir

def param_isnot_dir( pathdirname ):
    if not os.path.isdir( pathdirname ) :
        error_in_function( "'%s' is not a dir" % pathdirname, niv=2 )

10 Fichiers et répertoires

10.1 Obtenir le contenu d'un répertoire à la manière de Linux

import glob
for path in glob.glob("/home/C07138/.*")+glob.glob( "/home/C07138/*" ) : 
    print path

10.2 Liste des répertoires

#!/usr/bin/python
# -*- coding: ISO8859-1 -*-
import os, sys, re

def tofile( filename, text ) :
    f = open(filename, "w")
    f.write(text)
    f.close()

if __name__ == '__main__' :    
    fs0 = os.listdir('.')
    fs=list()
    for x in fs0 :
        fs.append(x.lower())
    
    fs.sort()
    s=''
    for x in fs :
        if os.path.isdir( x ) :
            s+=x.lower()+" "
    tofile( 'dir_list.txt', s )

10.3 Personnaliser "file object"

class myFileObject :
    def __init__( self, filename, mode="r" ) :
        self.f = open(filename,mode)

    def close(self) :
        return self.f.close()

    def read( self ) :
        txt = self.f.read().replace('\n\r','\n').replace('\r\n','\n').replace('\r','\n')
        return txt

    def write( self, txt ) :
        #txt = txt.replace('\n','\r')
        return self.f.write( txt )

def omypen( filename, mode="r" ) :
    return myFileObject( filename, mode )

10.4 Lire un fichier caractère par caractère

with open(filename,  "r", "UTF-8") as f:
    while True:
        c = f.read(1)
        if not c:
            print("End of file")
            break
        print("Read a character:", c)

10.5 Lire un fichier ligne par ligne

import codecs
ENCODING = "UTF-8"
with codecs.open(param.csvFileName, "r", ENCODING) as csvSrc:
    with codecs.open(csvDstName, "w", ENCODING) as csvDst:
        lineNumber = 0
        for line in csvSrc:
            line = line.replace("\n","").replace("\r","")
            lineNumber += 1

10.6 tofile fromfile

encoding = "UTF-8"
def toTextFile(filename, text):
    with open(filename, "w", encoding=encoding) as fileDesc:
        fileDesc.write(text)

def fromTextFile(filename):
    text = None
    with open(filename, "r", encoding=encoding) as fileDesc:
        text = fileDesc.read()
    return text


def fromTextFileWin(filename):
    text = None
    try : 
        with codecs.open(filename, "r", ENCODING) as fileDesc:
            text = fileDesc.read()
    except Exception as e :
        print("Warning %s" % e)
        print("try %s" % ENCODING2)
        with codecs.open(filename, "r", ENCODING2) as fileDesc:
            text = fileDesc.read()
    return text

10.7 Lire des fichiers ini

#!/usr/bin/python
# -*- coding: UTF-8 -*-
#***********************************************************
# Author:   Arthur TOROSSIAN
# Date:    30.10.2017
#***********************************************************/
import configparser
ABSTRACT_MSG = "call of abstract method"
################################################################################
class CIniFileError(Exception):
    """\
    the main exception class of the module
    """
    pass

def str2bool(v):
    v = v.strip().lower()
    if not v in ["true", "false", "vrai", "faux"]:
        raise CIniFileError("v='%s' is not a valid boolean" % v)
    return (v in ["true", "vrai"])

################################################################################
# CIniParam in order to read ini files
################################################################################
class CIniParam:
    ############################################################################
    def __init__(self):
        self.fileName = None
        self.config = None
        self._currentSection = None
    ############################################################################
    def loadParam(self, fileName):
        self.fileName = fileName
        self.config = configparser.ConfigParser()
        self.config.read(fileName)
        self.getParamFromConfig()

    def getParamFromConfig(self):
        """\
        (API) abstract required API method
        Check all options and attributes validity.
        """
        raise CIniFileError(ABSTRACT_MSG)

    def checkVersion(self, formatSection, v01, v02, v03):
        self.setCurrentSection(formatSection)
        v1 = self.getInt("v1")
        v2 = self.getInt("v2")
        v3 = self.getInt("v3")
        if v1 != v01 :
            errMsg = """

ERROR the file named '%s', in section named '%s', has no the right file format
version. The version must be %d.x.x and the current version is %d.%d.%d
""" % (self.fileName, formatSection, v01, v1, v2, v3)
            raise CIniFileError(errMsg)

    def hasTheSection(self):
        return self.config.has_section(self._currentSection)

    def hasOption(self, optionName):
        return self.config.has_option(self._currentSection, optionName)
    ############################################################################
    def setCurrentSection(self, currentSection):
        self._currentSection = currentSection
        if not self.hasTheSection():
            errMsg = """

ERROR the file named '%s', has no section named '%s'
""" % (
                    self.fileName, self._currentSection)
            raise CIniFileError(errMsg)


    def getFloat(self, varName):
        val = self.loadParamFloat(self._currentSection, varName)
        return val

    def getFloatList(self, varName):
        valList = eval(self.loadParamStr(self._currentSection, varName))
        newVal = list()
        for sf in valList :
            newVal.append(float(sf))
        return newVal

    def getFloatOrNoneList(self, varName):
        valList = eval(self.loadParamStr(self._currentSection, varName))
        newVal = list()
        for sf in valList :
            if sf == None :
                newVal.append(sf)
            else:
                newVal.append(float(sf))
        return newVal

    def getStrList(self, varName):
        valList = eval(self.loadParamStr(self._currentSection, varName))
        newVal = list()
        for sf in valList :
            newVal.append(sf)
        return newVal

    def getInt(self, varName):
        val = self.loadParamInt(self._currentSection, varName)
        return val

    def getBool(self, varName):
        val = self.loadParamBool(self._currentSection, varName)
        return val

    def getStr(self, varName):
        val = self.loadParamStr(self._currentSection, varName)
        return val

    ############################################################################
    def checkNoParam(self, paramName):
        if not self.hasOption(paramName):
            errMsg = """

ERROR the file named '%s', in section named '%s', has no parameter named '%s'
""" % (self.fileName, self._currentSection, paramName)
            raise CIniFileError(errMsg)

    def loadParamFloat(self, sectionName, varName):
        self.checkNoParam(varName)
        xs = self.config.get(sectionName, varName).split("#")[0]
        x = float(eval(xs))
        return x

    def loadParamInt(self, sectionName, varName):
        self.checkNoParam(varName)
        x = int(self.config.get(sectionName, varName).split("#")[0])
        return x

    def loadParamBool(self, sectionName, varName):
        self.checkNoParam(varName)
        dat = self.config.get(sectionName, varName)
        x = str2bool(dat.split("#")[0])
        return x

    def loadParamStr(self, sectionName, varName):
        self.checkNoParam(varName)
        x = self.config.get(sectionName, varName).split("#")[0]
        return x

class CParams(CIniParam) :
    def getParamFromConfig(pm):
        pm.setCurrentSection("param")
        pm.tMax = pm.getFloat("p1")

10.8 fichiers temporaires, tmp, temp

tmp_file=tempfile.NamedTemporaryFile( delete=False )
tmp_file.close()
print tmp_file.name
print os.path.exists(tmp_file.name)
tgzs=fromfile( tmp_file.name )
s.unlink(tmp_file.name)
print os.path.exists(tmp_file.name)

10.9 Lire et écrire des binaires (endianness; little-endian et big-endian) dans des fichiers

On utilise le module struct avec les méthodes unpack et pack :

def scanDatFile( fileName, fileDestinationName, endianness='<', ee1WaterDeep=5, ee1LandHeight=10 ) :
    bs = os.path.getsize( fileName ) 
    fsize = 0    
    fic = open( fileName , 'rb')
    ficDest = open( fileDestinationName , 'wb')
    # < little-endian BYTEORDER LSBFIRST
    # > big-endian
    # h short integer 2 bytes 
    fmt = "%sh" % endianness
    min = 60000
    max = -60000
    noval=-32768
    while fsize < bs :
        fsize += 2
        tmp = fic.read( struct.calcsize(fmt) )
        val, = struct.unpack( fmt , tmp  )    
        valBin = struct.pack( fmt , func.hcorection( val )  )   
        ficDest.write( valBin )        
        #print val
        if val != noval :
            if val < min :
                min = val
            if val > max :
                max = val
    print min, max
    ee1 = ee1WaterDeep + ee1LandHeight
    map = float(max-min+1)
    ee2map = map / ee1
    ee1WaterDeep = round( min/ee2map )
    ee1LandHeight = round( max/ee2map )
    print "Water Deep:%s Land height: %s" %( round( min/ee2map ), round( max/ee2map )  )
    print "Water Cutoff:%s Land Cutoff: %s" %( round( ee1WaterDeep*ee2map ), round( ee1LandHeight*ee2map )  )
    ficDest.close()
    fic.close()

10.10 Lire et écrire des fichiers netCDF

• Read and Write netCDF Files

  https://pythonhosted.org/landlab/landlab.io.netcdf.html

10.11 Copier les dates d'un fichier à un autre

st=os.stat( file1 )
os.utime( file2, (st.st_atime, st.st_mtime) )

10.12 trace

def get_datetime_str():
    return time.strftime("%Y-%m-%d_%H-%M-%S",time.localtime())

trace_file_name="trace.txt"
def trace( msg, init=0 ) :
    filename = trace_file_name
    if (os.path.exists( filename ) and (init==0) ):
        f = open( filename, "a"  )
    else :
        f = open( filename, "w"  )
    f.write( "%s %s\n" % (get_datetime_str(),msg) )
    f.close()
trace( "INIT", init=1 )

10.13 get_NewFileName

def get_NewFileName( name, Index=1 ) :
    newname = name    
    name, ext = os.path.splitext( newname  )
    while os.path.exists( newname ) :        
        newname = name + ("%03d" % Index) + ext
        Index += 1
    return newname, Index

10.14 get_NewDictName

patName = re.compile("^(.*?)(\d*)$")
def normName( name  ) :
    nname, ni = patName.findall( name )[0]
    if ni != "" :
        i = int( ni )
        newname = nname + ("%03d" % i)

    else :
        newname = name
    return newname

def normDictNames( namedict ) :
    newD = dict()
    for name, val in namedict.items() :
        newname = normName( name  )
        newD[ newname ] = val
    return newD

def get_NewDictName( name, namedict, Index=1 ) :
    newname = name
    name, i = patName.findall( newname )[0]
    while newname in namedict :
        newname = name + ("%03d" % Index)
        Index += 1
    return newname, Index

utilisation possible  :

dd0 = {"otto":1,"toto3":2, "toto02":4, "tata":5, "toto" :1 }
print dd0
dd = normDictNames( dd0 )
print dd
name, index = get_NewDictName( "toto", dd  )
dd[name] = 30
print name, index, dd
name, index = get_NewDictName( "toto", dd  )
dd[name] = 15
print name, index, dd

{'tata': 5, 'toto': 1, 'toto02': 4, 'toto3': 2, 'otto': 1}
{'toto': 1, 'tata': 5, 'toto002': 4, 'toto003': 2, 'otto': 1}
toto001 2 {'toto003': 2, 'tata': 5, 'toto': 1, 'toto002': 4, 'toto001': 30, 'otto': 1}
toto004 5 {'toto003': 2, 'tata': 5, 'toto004': 15, 'toto': 1, 'toto002': 4, 'toto001': 30, 'otto': 1}

10.15 path_split

def path_split( pathnameext ):
    path, name = os.path.split( pathnameext )
    name , ext = os.path.splitext( name )
    return (path, name, ext,)

10.16 change_path_and_suffix

def change_path_and_suffix(  pathnameext, newpath, suffix ) :
    path, nameext = os.path.split( pathnameext )
    name, ext = os.path.splitext( nameext )
    return newpath + os.sep + name+suffix+ext

10.17 change_path_and_ext

def change_path_and_ext(  pathnameext, newpath, ext ) :
    path, nameext = os.path.split( pathnameext )
    name, ext0 = os.path.splitext( nameext )
    return newpath + os.sep + name+ext

10.18 Get list of files

Simple :

def findFilesInDirectorFromPattern( path, pat ) :
    flist = list() 
    fs = os.listdir(path)
    for x in fs :
        file_path = path+os.sep+x 
        if os.path.isfile( file_path  ) :
            if pat.match( x ) :
                flist.append( file_path  )
    return flist

Évoluée :

def find( flist, path, file_pattern,  subdir, file_exclude_pattern=None, dir_exclude_pattern=None ) :
    fs = os.listdir(path)
    for x in fs :
        file_path = path+os.sep+x
 
        if os.path.isfile( file_path  ) :
            ok=1
            if file_exclude_pattern != None :
                ok = not file_exclude_pattern.match( x )
            if ok and file_pattern.match( x ) :
                print x
                flist.append( file_path  )
 
        if os.path.isdir( file_path  ) :
            if subdir :
                ok = 1
                if dir_exclude_pattern != None :
                    ok= not dir_exclude_pattern.match( x )
                if ok :
                    find( flist, file_path, file_pattern,  subdir, file_exclude_pattern, dir_exclude_pattern)

10.19 Lire et écrire des fichiers HTML

Il ne faut pas utiliser html.parser, Simple HTML ou XHTML parser mais il est largement recommandé d'utiliser sgmllib ou Simple SGML parser.

10.20 Générer une fichier html à partir d'une liste de fichiers sur disques gen_list_html

#!/usr/bin/env python
import re
import os
def gen_list_html( path, file_pattern_txt ) :
    file_pattern = re.compile( file_pattern_txt )
    fs = os.listdir(path)
    fs.sort()
    print "<html><body><ol>"
    for x in fs :
        file_path = path+os.sep+x
        if os.path.isfile( file_path  ) :
            if file_pattern.match( x ) :
                print "<li><a href='%s'>%s</a></li>" %(x,x)

    print "</ol></body></html>"
if __name__ == '__main__' :
    gen_list_html( ".", ".*\.avi$" )

10.21 Générer une fichier html à partir d'une liste d'images/photo sur disques

#!/usr/bin/python
# -*- coding: UTF-8 -*-
import os, sys, re, codecs, time, shutil


srcPath = "."
fl = os.listdir(srcPath)
sorted(fl)

f = codecs.open("index.html", "w", encoding = "UTF-8")

f.write("<html style='background-color:#a0a0a0;margin:auto;'>\n")

f.write("""<body 
style='width:210mm;margin:auto;padding:5mm;background-color:#ffffff;'>
<style type="text/css">
p.p {
width:180mm;background-color:#ffffff;margin:auto;
}
td.g {
width:30mm; vertical-align:middle;
}
td.d {
width:150mm; vertical-align:top; background-color:#ffffff;font-size:10pt;
}
img{
width:28mm;
}
</style>
<p class="p">
<table align=center>
""")

for img in fl :
    name, ext = os.path.splitext(img)
    print(name.lower())
    if ext.lower() == ".jpg" :
        f.write("""\
<tr>
    <td class='g'> <img src='%s'> </td>
    <td class='d'> TODO </td>
</tr>
""" % img)

f.write("""
</table>
</p>
</body>
""")
f.write("</html>\n")

f.close()

10.22 gen m3u8 - liste de mp3 - player list - utilisable par Rhythmbox

voir playList (svn atTools)

10.23 genNewNames

#!/usr/bin/python
# -*- coding: UTF-8 -*-
import os, sys, re, random

def get_NewFileName( name, Index=1 ) :
    newname = name    
    name, ext = os.path.splitext( newname  )
    while os.path.exists( newname ) :        
        newname = name + ("%03d" % Index) + ext
        Index += 1
    return newname, Index

def genNewNames( path ) :
    index=1
    fs = os.listdir(path)
    nb = len(fs)
    num = 0
    while nb > 0 :
        i = random.randint(0, nb-1 )
        nname = path+os.sep+fs[i]
        del fs[i]
        if os.path.isfile( nname ) :
            nname0, ext = os.path.splitext(nname)
            pname, nname0 = os.path.split(nname)
            rname = pname + os.sep+"im%05d%s" % (num,ext)
            print "  ",nname, rname
            rname, index = get_NewFileName( rname, index )
            os.rename( nname , rname )        
        num += 1
        nb -= 1
        
def genNewNamesInSubDirs( path ) :    
    fs = os.listdir(path)
    for ff in fs :
        pathff = path+os.sep+ff
        print "[%s]" % pathff
        if os.path.isdir( pathff ) :
            genNewNames( pathff )
if __name__ == u'__main__' :    
    genNewNamesInSubDirs( "./Images/listes")   
    genNewNames( "./Images/listeAll" )

10.24 unix2dos

#!/usr/bin/env python
###############
import sys

for fname in sys.argv[1:]:
    infile = open( fname, "rb" )
    instr = infile.read()
    infile.close()
    outstr = instr.replace( "\r\n", "\n" ).replace( "\n\r", "\n" ).replace( "\r", "\n" ).replace( "\n", "\r\n" )

    if len(outstr) == len(instr):
        continue
    
    outfile = open( fname, "wb" )
    outfile.write( outstr )
    outfile.close()

10.25 clean_rep.py

#!/usr/bin/env python
# -*- coding: iso-8859-1 -*-
import os
import re
pat_adir = re.compile( "\d{4}" )
pat_mdir = re.compile( "\d{4}-\d{2}" )
def clean_dirs() :
    """
    Delete empty directories with name like AAAA/AAAA-MM or AAAA.
    """
    adirs = os.listdir(".")
    for adir in adirs :
        if pat_adir.match( adir ) :
            mdirs = os.listdir(adir)
            for mdir in mdirs :
                if pat_mdir.match( mdir ) :
                    files = os.listdir( adir+os.sep+mdir )
                    if len(files)==0 :
                        os.rmdir( adir+os.sep+mdir )
            mdirs = os.listdir(adir)
            if len( mdirs )==0 :
                os.rmdir( adir )

if __name__ == '__main__' :
    clean_dirs()

10.26 gen_rep.py

#!/usr/bin/env python
# -*- coding: iso-8859-1 -*-

import os

def gen_dirs() :
    """
    Generate directory with name like AAAA/AAAA-MM if it does not exist.
    """
    for an in range(2000,2012) :
        dir_a = "%04d" %(an)
        if not os.path.exists( dir_a ) :
            os.mkdir( dir_a )
        for m in range(1,13) :
            dir_name = "%s%s%04d-%02d" %(dir_a,os.sep,an,m)
            print dir_name
            if not os.path.exists( dir_name ) :
                os.mkdir( dir_name )

if __name__ == '__main__' :
    gen_dirs()

10.27 size_on_system

def size_on_system( sf, sector_size ):
    if sector_size != None :
        s=0
        ns = sf % sector_size
        if ns == 0 :
            s = sf
        else :
            s = (sf / sector_size) * sector_size  + sector_size
    else :
      s=sf
    return s

10.28 size_of_dir

def size_of_dir( path, sector_size=None  ):
    fs = os.listdir( path )
    s=0
    for f in fs :
        pf = path+os.sep+f
        if os.path.isfile( pf ) :
            statinfo = os.stat( pf )
            s+= size_on_system( statinfo.st_size, sector_size )
        if os.path.isdir( pf ) :
            s+=size_of_dir( pf,sector_size  )
    return s

10.29 sizeOfFile

def sizeOfFile( pathToFile ):
    size = None
    if os.path.isfile( pathToFile ) :
        statinfo = os.stat( pathToFile )
        size = statinfo.st_size
    return size

10.30 have_subdir

def have_subdir( path ) :
    ok = 0
    fs = os.listdir( path )
    nb = len( fs )
    i = 0
    while (i < nb) and (ok==0) :
        pathx = path + os.sep + fs[ i ]
        if os.path.isdir( pathx ) :
            ok = 1
        i += 1
    return ok

11 Compte utilisateur

11.1 get_username

def get_username():
    return getpass.getuser()

11.2 get_hostname

def get_hostname():
    return socket.gethostbyaddr(socket.gethostname())[0]

11.3 get_homedir

def get_homedir():
    homedir = os.path.expanduser('~')
    return homedir

12 Réseau, Internet

12.1 fromurl

def fromurl( cmd ) :
    encore = 1
    while encore :
        try :    
            f = urllib.urlopen( cmd )
            rep = f.read()
            f.close()
            encore = 0
        except Exception,e :
            #print str(e)
            encore = 1
            
    return rep

• urrlib2 opener for SSL proxy (CONNECT method) (Python recipe)
• How to use m2crypto

12.2 socket.setdefaulttimeout

    timeout = 10
    socket.setdefaulttimeout(timeout)
    text = urllib2.urlopen('http://at.bht.fr').read()
    print( text )

12.3 HTTPBasicAuthHandler

    auth_handler = urllib2.HTTPBasicAuthHandler()
    auth_handler.add_password(
                          realm='My Application',
                          uri='',
                          user='',
                          passwd=''
                          )
    opener = urllib2.build_opener( auth_handler )
    # ...and install it globally so it can be used with urlopen.
    urllib2.install_opener( opener )    
    text = urllib2.urlopen('https://at.bht.fr/DisEvtSim').read()
    print( text )

12.4 ProxyBasicAuthHandler

    proxy_handler = urllib2.ProxyHandler({'http': 'http://proxypac.edf.fr:3128/','https': 'https://proxypac.edf.fr:3128/'})
    proxy_auth_handler = urllib2.ProxyBasicAuthHandler()
    proxy_auth_handler.add_password('realm', 'proxypac.edf.fr', '', '')
    opener = urllib2.build_opener(proxy_handler, proxy_auth_handler)    
    # ...and install it globally so it can be used with urlopen.
    urllib2.install_opener(opener)    
    text = urllib2.urlopen('http://at.bht.fr').read()
    print( text )

12.5 ProxyBasicAuthHandler

    proxy_handler = urllib2.ProxyHandler({'http': 'http://proxypac.edf.fr:3128/','https': 'https://proxypac.edf.fr:3128/'})
    proxy_auth_handler = urllib2.ProxyBasicAuthHandler()
    proxy_auth_handler.add_password('realm', 'proxypac.edf.fr', '', '')
    auth_handler = urllib2.HTTPBasicAuthHandler()
    auth_handler.add_password(realm='PDQ Application',
                          uri='https://at.bht.fr/DisEvtSim',
                          user='',
                          passwd='')
    
    opener = urllib2.build_opener(proxy_handler, proxy_auth_handler,auth_handler)    
    # ...and install it globally so it can be used with urlopen.
    urllib2.install_opener(opener)    
    text = urllib2.urlopen('https://at.bht.fr/DisEvtSim').read()
    print( text )

12.6 https sans authentification

    timeout = 10
    socket.setdefaulttimeout(timeout)    
    text = urllib2.urlopen('https://www.google.fr').read()
    print( text )

13 Expressions régulières

• voir plus généralement Expressions régulières

13.1 Never match

class CNeverMatch :
    def match( self, f ) :
        return 0

13.2 Allways match

class CAllwaysMatch :
    def match( self, f ) :
        return 1

13.3 Remplacer avec sub : replace_re.py

Un petit programme complet très utile pour traiter les fichiers (utilisation de sub)

#!/usr/bin/python
# -*- coding: 'ISO8859-1' -*-
import os, sys, re

def fromfile( filename ) :
    f = open(filename, "r")
    text = f.read()
    f.close()
    return text

def replace_re( exp, remplacant, txt ) :
    pat=re.compile( exp )
    newtxt= re.sub( pat, remplacant, txt) 
    return newtxt

def replace_re_file( exp, remplacant, filename ) :
    txt = fromfile( filename )
    return replace_re( exp, remplacant, txt ) 

def help() :
    print "HELP"
    print "\treplace_re.py expression_reguliere remplacant filename1 "

if __name__ == '__main__' :    
    if len( sys.argv  ) == 4 :
        txt = replace_re_file( sys.argv[1], sys.argv[2], sys.argv[3] )
        print txt
    else :
        help()

13.4 findall et sub

Le code suivant remplace dans les lignes d'un fichier telles que

<td style="cursor:pointer; background-color: rgb(255, 255, 153); border: 1px solid #CCC;">&nbsp;</a></td>

toutes le chaines &nbsp; par le calcul des couleurs qui se trouvent dans rgb(XXX, XXX, XXX), cela donne dans notre cas :

<td style="cursor:pointer; background-color: rgb(255, 255, 153); border: 1px solid #CCC;" >#ffff99</td>

def rgb( r,v,b ) :
    return "#%02x%02x%02x" %(r,v,b)

def dd( filename ) :
    pattern=re.compile(".*(rgb\(\d+, *\d+, *\d+\));.*")
    pat = re.compile( " " )
    txt = fromfile( filename ).split("\n")
    newtxt=""
    for l in txt :
        tr=re.findall( pattern, l) 
        if len(tr)>0 :
            code=eval(tr[0])
            newl= re.sub( pat, code, l)
        else :
            newl=l
        print newl
        newtxt+=newl

14 Hashage

14.1 crc32 : hashage de lignes de texte ou de binaires

#!/usr/bin/python
#===============================================================================
# -*- coding: UTF-8 -*-
#===============================================================================
import binascii

print("b:")
print( " ", b"hello" )
print( " ", "crc32: %0x" % binascii.crc32(b"hello") )  # bad way -> bytearray
print("bytearray:")
print( " ", bytearray("hello", "utf8" ) )
print( " ", "crc32: %0x" % binascii.crc32( bytearray("hello", "utf8" )))
print("b:")
print( " ", b"barba-papa" )
print( " ", "crc32: %0x" % binascii.crc32 (b"barba-papa" ) )  # bad way -> bytearray
print("bytearray:")
print( " ", bytearray("barba-papa","utf8" ) )
print( " ", "crc32: %0x" % binascii.crc32(  bytearray("barba-papa", "utf8" ) ) )
print()
print( " ", bytearray("Installation on Linux (Obsolète)", "utf8" ) )
print( " ", "crc32: %0x" % binascii.crc32( bytearray("Installation on Linux (Obsolète)", "utf8" )))

15 Les conversions

15.1 Limiter le nombre de chiffres significatifs

def roundToSignificantDigitNumber( x , nbDigit ):
    if x != 0.0 :
        ten = round(math.log10(x))
        tenFac = 10 ** ten
        x = round(x / tenFac,  nbDigit-1)
        x = x * tenFac
    return x

15.2 Convertir les None en nan

On utilise les liste en compréhension :

a = ( None, 1.2, -4 )
b = [ x  if x is not None else float("nan") for x in a ]

on obtient:

[nan, 1.2, -4]

15.3 hexadécimal

def hextoint( x ):
    return int(x,16)
def inttohex( x ):
    return hex(x)

15.4 binaire

a= "{0:08b}".format( a )

ou

def bintoint( x ):
    return int(x,2)

def inttobin( x ):
    return bin(x)

15.5 Couleur vers int (alpha, red, green, blue)

def argbToInt( a, r, g, b ):
    st=struct.pack(">BBBB", a, r, g, b )
    i= struct.unpack(">I", st)[0] 
    return i

def intToArgb( i ):
    print i
    st=struct.pack(">I", i ) 
    return struct.unpack(">BBBB", st)

16 Formatage de texte

16.1 Le principe de la méthode .format

Un exemple pour comprendre rapidement :

"{name1:1.2f} bla bla {name2:02d} {name1:f}".format(name1=3.14, name2=456)

Ce qui produit :

'3.14 bla bla 456 3.140000'

17 Physique

17.1 Nombre de chiffres significatifs

voir Comment limiter le nombre de chiffres significatifs.

17.2 Rationnels et calculs symboliques / formels

• What is Symbolic Computation?
• Number Theory in SymPy
• sympy
• mpmath
• Welcome to mpmath’s documentation!

from sympy import Rational
a = Rational(2,3)
print(a)
b = Rational(4,3)
print(a+b)

a=Rational(2*3*5*7*7,2*7*31)
print(a)

sympy.factorint(105)
a = 2*3*5*7*7
b= 2*7*31
sympy.factorint(a)
sympy.factorint(b)
pgcd = sympy.gcd(a,b)
print(pgcd)

aa= a / pgcd
bb= b / pgcd
sympy.factorint(aa)
sympy.factorint(bb)

2/3

2

105
───
 31

{3: 1, 5: 1, 7: 1}
{2: 1, 3: 1, 5: 1, 7: 2}
{2: 1, 7: 1, 31: 1}

14

{3: 1, 5: 1, 7: 1}
{31: 1}

from sympy import symbol
from sympy import symbols
x = symbols("x")
sympy.init_printing(use_unicode=True)
sympy.integrate(sympy.sin(x**2), x)
sympy.integrate(sympy.sin(x**2), (x,-sympy.oo,sympy.oo))

Intégrale :

                ⎛√2⋅x⎞       
3⋅√2⋅√π⋅fresnels⎜────⎟⋅Γ(3/4)
                ⎝ √π ⎠       
─────────────────────────────
           8⋅Γ(7/4)          

√2⋅√π
─────
  2

17.3 Unités

• Voir [1]

from sympy.physics.units import *
a = 254 * m / s
b =  convert_to(a, [minute])
print(b)
find_unit(kg)

15240*meter/minute

[g, kg, mg, ug, amu, mmu, amus, gram, mmus, grams, pound, pounds, 
kilogram, kilograms, microgram, milligram, micrograms, milligrams, 
planck_mass, milli_mass_unit, atomic_mass_unit, atomic_mass_constant]

17.3.1 Listes des unités

# Dimensionless

percent = percents = Rational(1, 100)
permille = permille = Rational(1, 1000)

ten = Rational(10)

yotta = ten**24
zetta = ten**21
exa = ten**18
peta = ten**15
tera = ten**12
giga = ten**9
mega = ten**6
kilo = ten**3
deca = ten**1
deci = ten**-1
centi = ten**-2
milli = ten**-3
micro = ten**-6
nano = ten**-9
pico = ten**-12
femto = ten**-15
atto = ten**-18
zepto = ten**-21
yocto = ten**-24

rad = radian = radians = 1
deg = degree = degrees = pi/180
sr = steradian = steradians = 1

# Base units

length = m = meter = meters = Unit('meter', 'm')
mass = kg = kilogram = kilograms = Unit('kilogram', 'kg')
time = s = second = seconds = Unit('second', 's')
current = A = ampere = amperes = Unit('ampere', 'A')
temperature = K = kelvin = kelvins = Unit('kelvin', 'K')
amount = mol = mole = moles = Unit('mole', 'mol')
luminosity = cd = candela = candelas = Unit('candela', 'cd')


# Derived units
volume = meter**3
frequency = Hz = hz = hertz = 1/s
force = N = newton = newtons = m*kg/s**2
energy = J = joule = joules = N*m
power = W = watt = watts = J/s
pressure = Pa = pa = pascal = pascals = N/m**2
charge = C = coulomb = coulombs = s*A
voltage = v = V = volt = volts = W/A
resistance = ohm = ohms = V/A
conductance = S = siemens = mho = mhos = A/V
capacitance = F = farad = farads = C/V
magnetic_flux = Wb = wb = weber = webers = J/A
magnetic_flux_density = T = tesla = teslas = V*s/m**2
inductance = H = henry = henrys = V*s/A
speed = m/s
acceleration = m/s**2
density = kg/m**3
optical_power = dioptre = D = 1/m
illuminance = lux = lx = sr*cd/m**2

# Common length units

km = kilometer = kilometers = kilo*m
dm = decimeter = decimeters = deci*m
cm = centimeter = centimeters = centi*m
mm = millimeter = millimeters = milli*m
um = micrometer = micrometers = micron = microns = micro*m
nm = nanometer = nanometers = nano*m
pm = picometer = picometers = pico*m

ft = foot = feet = Rational('0.3048')*m
inch = inches = Rational('25.4')*mm
yd = yard = yards = 3*ft
mi = mile = miles = 5280*ft


# Common volume and area units

l = liter = liters = m**3 / 1000
dl = deciliter = deciliters = deci*l
cl = centiliter = centiliters = centi*l
ml = milliliter = milliliters = milli*l


# Common time units

ms = millisecond = milliseconds = milli*s
us = microsecond = microseconds = micro*s
ns = nanosecond = nanoseconds = nano*s
ps = picosecond = picoseconds = pico*s

minute = minutes = 60*s
h = hour = hours = 60*minute
day = days = 24*hour

anomalistic_year = anomalistic_years = Rational('365.259636')*day
sidereal_year = sidereal_years = Rational('31558149.540')*s
tropical_year = tropical_years = Rational('365.24219')*day
common_year = common_years = Rational('365')*day
julian_year = julian_years = Rational('365.25')*day
draconic_year = draconic_years = Rational('346.62')*day
gaussian_year = gaussian_years = Rational('365.2568983')*day
full_moon_cycle = full_moon_cycles = Rational('411.78443029')*day

year = years = tropical_year


# Common mass units

g = gram = grams = kilogram / kilo
mg = milligram = milligrams = milli * g
ug = microgram = micrograms = micro * g


#----------------------------------------------------------------------------
# Physical constants
#
c = speed_of_light = 299792458 * m/s
G = gravitational_constant = Rational('6.67428') * ten**-11 * m**3 / kg / s**2
u0 = magnetic_constant = 4*pi * ten**-7 * N/A**2
e0 = electric_constant = 1/(u0 * c**2)
Z0 = vacuum_impedance = u0 * c

planck = Rational('6.62606896') * ten**-34 * J*s
hbar = planck / (2*pi)

avogadro_number = Rational('6.02214179') * 10**23
avogadro = avogadro_constant = avogadro_number / mol
boltzmann = Rational('1.3806505') * ten**-23 * J / K

gee = gees = Rational('9.80665') * m/s**2
atmosphere = atmospheres = atm = 101325 * pascal

kPa = kilo*Pa
bar = bars = 100*kPa
pound = pounds = 0.45359237 * kg * gee  # exact
psi = pound / inch ** 2
dHg0 = 13.5951  # approx value at 0 C
mmHg = dHg0 * 9.80665 * Pa
amu = amus = gram / avogadro / mol
mmu = mmus = gram / mol
quart = quarts = Rational(231, 4) * inch**3
eV = 1.602176487e-19 * J

# Other convenient units and magnitudes

ly = lightyear = lightyears = c*julian_year
au = astronomical_unit = astronomical_units = 149597870691*m

18 Math

18.1 Équation de second degré

def secondDegreeEquation(a,b,c):
    # a*x*x + b*y + c = 0
    x1 = None
    x2 = None
    det = b*b - 4 * a * c
    if det>= 0.0 :
        sqDet = math.sqrt(det)
        a2 = a * 2.0
        x1 = (-b - sqDet ) / a2 
        x2 = (-b + sqDet ) / a2
    return x1, x2

18.2 Régression linéaire

source www.science-emergence.com

Fichier:LinearRegressionInPython.png

Fig. n°5: Régression linéaire

from sklearn import linear_model
import numpy as np
import matplotlib.pyplot as plt

x,y = np.loadtxt("test.txt", unpack='true')

plt.scatter(x,y)

regr = linear_model.LinearRegression()
regr.fit(x[:,np.newaxis], y)

x_test = np.linspace(np.min(x), np.max(x), 100)

plt.plot(x_test, regr.predict(x_test[:,np.newaxis]), color='blue', linewidth=3)

plt.show()

Pour obtenir le coefficient directeur et l'ordonnée à l'origine de la droite:

print 'slope', regr.coef_
print 'intercept', regr.intercept_

donne ici

slope [ 0.80723367]
intercept -0.623011727604

18.3 Décomposition d'un nombre en facteurs premiers

#!/usr/bin/python
# -*- coding: utf-8 -*-
 
def facteurs(n):
    """facteurs(n): décomposition d'un nombre entier n en facteurs premiers"""
    F = []
    if n==1:
        return F
    # recherche de tous les facteurs 2 s'il y en a
    while n>=2:
        x,r = divmod(n,2)
        if r!=0:
            break
        F.append(2)
        n = x
    # recherche des facteurs 1er >2
    i=3
    rn = lsqrt(n)+1
    while i<=n:
        if i>rn:
            F.append(n)
            break
        x,r = divmod(n,i)
        if r==0:
            F.append(i)
            n=x
            rn = lsqrt(n)+1
        else:
            i += 2
    return F
 
# exemple d'utilisation:
print facteurs(100)  # affiche [2,2,5,5]
print facteurs(123456789)  # affiche [3,3,3607,3803]
print facteurs(12345678901234567890) # affiche [2,3,3,5,101,3541,3607,3803,27961L]
 
# et test avec un nombre premier
print facteurs(4291979)  # affiche [4291979]

source: http://python.jpvweb.com/python/mesrecettespython/doku.php?id=decomposition_en_facteurs_premiers

18.4 Histogram - densité de probabilité

18.4.1 Numpy

Utilisation la fonction d’histogramme de numpy voir [2]

Avec l’option {{{1}}} :

Soit n Intervalles I(i) de longueur L(i) et Q(i) la quantité de valeurs incluse dans I(i)

Soit Qt = somme des Qi

Alors l’histogramme avec density=True est la suite des H(i) = Q(i)/L(i)/Qt

Et ainsi l’intégrale de de l’histogramme vaut 1 car c’est la somme de des H(i)*L(i) = somme de Q(i)/Qt = 1

Voici un petit code python pour illustrer :

#!/usr/bin/python
# -*- coding: UTF-8 -*-
import numpy as np
interval = np.linspace(-0.5,1.5,20)
print(interval)
intervalD = np.diff(interval)
print(intervalD)
vals = [ -0.2, 0.1, 0.4, -0.8, 2.8, 0.4 , 6.8, 1.3  ]
hist0 = np.histogram( vals, bins=interval )
hist = np.histogram(  vals, bins=interval , density=True)
print("quantity")
print(hist0[0].tolist())
print("density")
print(hist[0].tolist())
print("integral")
print( np.sum(hist[0]*intervalD)  )

ce qui donne :

[-0.5        -0.39473684 -0.28947368 -0.18421053 -0.07894737  0.02631579
  0.13157895  0.23684211  0.34210526  0.44736842  0.55263158  0.65789474
  0.76315789  0.86842105  0.97368421  1.07894737  1.18421053  1.28947368
  1.39473684  1.5       ]
[0.10526316 0.10526316 0.10526316 0.10526316 0.10526316 0.10526316
0.10526316 0.10526316 0.10526316 0.10526316 0.10526316 0.10526316
0.10526316 0.10526316 0.10526316 0.10526316 0.10526316 0.10526316
0.10526316]
quantity
[0, 0, 1, 0, 0, 1, 0, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0]
density
[0.0, 0.0, 1.9, 0.0, 0.0, 1.9, 0.0, 0.0, 3.8, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.8999999999999981, 0.0]
integral
1.0

19 Parallélisme

19.1 multiprocessing

#!/usr/bin/python
# -*- coding: UTF-8 -*-
#***********************************************************

import time
import math
import sys
from multiprocessing import Pool

def getPoreccessorNumber():
    return multiprocessing.cpu_count()

def parallelCall(params):  # a helper for calling 'remote' instances
    
    # get our class type
    cls = getattr(sys.modules[__name__], params[0])  
    
    # create a new instance without invoking __init__
    instance = cls.__new__(cls)  
    
    # apply the passed state to the new instance
    instance.__dict__ = params[1]  
    
    # get the requested method
    method = getattr(instance, params[2])  
    args = params[3] if isinstance(params[3], (list, tuple)) else [params[3]]
    
    # expand arguments, call our method and return the result
    return method(*args)  

class CMultiProc(object):
    def __init__(self):
        self.val = None

    def calc(self, num,  i1, i2):
        self.num = num        
        val = 0        
        for i in range(i1, i2) :
                #val += math.cos(math.sqrt(i*i))
                val += i*i
        self.val = val
        return ( self.num, val )

    def do(self, data):
        t = Pool(processes=len(data))
        rs = t.map(parallelCall, self.prepareCall("calc", (data) ))
        t.close()
        return rs

    # creates a 'remote call' package for each argument
    def prepareCall(self, name, args):  
        for arg in args:
            yield [self.__class__.__name__, self.__dict__, name, arg]

if __name__ == "__main__":  # important protection for cross-platform use
    
    ############################################################################
    data = [ 
            (1,0       ,10000000), 
            (2,10000000,20000000), 
            (3,20000000,30000000), 
            (4,30000000,40000000), 
            (5,40000000,50000000),
            (6,50000000,60000000), 
            (7,60000000,70000000), 
            (8,70000000,80000000), 
            (9,80000000,90000000), 
            (10,90000000,100000000), 
            (11,100000000,110000000),
            (12,110000000,120000000),
            (13,120000000,130000000),
            (14,130000000,140000000),
            (15,140000000,150000000),
            (16,150000000,160000000),  
            ]
    ############################################################################
    # parallel    
    print("parallèle")
    t0 = time.time()
    multiProc = CMultiProc(  )    
    res =  multiProc.do( data  )
    vp = 0
    for num, val in res :
        vp += val
        print("% 5d % 5.1f " % (num, val) )
    print(vp)                
    pt = time.time()-t0           
    print("t=% 5.1f" % (pt),"s" )
    ############################################################################    
    # serial
    print("série")
    t0 = time.time()
    vs = 0
    for i in range(0,160000000) :
        # v += math.cos((math.sqrt(i*i)) )
        vs += i*i
    st = time.time()-t0
    print(vs)    
    print("t=% 5.1f" % (st),"s" )
    ############################################################################    
    # diff
    print("Diff", vp -vs )
    print("Gain", st / pt  )

Résultats:

parallèle
    1  333333283333335023616.0
    2  2333333183333335040000.0
    3  6333333083333335187456.0
    4  12333332983333334810624.0
    5  20333332883333336530944.0
    6  30333332783333336154112.0
    7  42333332683333335777280.0
    8  56333332583333331206144.0
    9  72333332483333330829312.0
   10  90333332383333334646784.0
   11  110333332283333334269952.0
   12  132333332183333329698816.0
   13  156333332083333337710592.0
   14  182333331983333341528064.0
   15  210333331883333324374016.0
   16  240333331783333319802880.0
1365333320533333360000000
t=  1.9 s
série
1365333320533333360000000
t= 12.0 s
Diff 0
Gain 6.214744302379906

19.2 Timer (onsolète voir multiprocessing)

import threading
prLock = threading.RLock()
# ...
        # initialization
        self.timer = None
        self.executing = False
        self.wating = False
# ...
    ############################################################################
    def doSomeThing(self):
        global prLock
        prLock.acquire()
        if self.executing:

            # ... do someThing in parallel thread

        prLock.release()
        self.launchTimer()
    ############################################################################
    def launchTimer(self):
        global prLock
        if self.executing:
            self.timer = threading.Timer(10, self.doSomeThing)
            self.timer.start()
        else:
            prLock.acquire()
            self.wating = False
            prLock.release()
# ...
    # start the main process
    prLock.acquire()
    param.wating = True
    param.executing = True
    prLock.release()

    # ... do things in the main thread

    prLock.acquire()
    param.executing = False

    print "wait ..."
    prLock.release()

    waiting = True
    while waiting:
        prLock.acquire()
        waiting = param.wating
        prLock.release()
    # the end of all

20 Aller plus loin avec le langage

20.1 Obtenir le nom ou les noms de la variable

tdo-1

A tester et valider :

def find_names(obj):
    frame = sys._getframe()
    for frame in iter(lambda: frame.f_back, None):
        frame.f_locals
    result = []
    for referrer in gc.get_referrers(obj):
        if isinstance(referrer, dict):
            for k, v in referrer.iteritems():
                if v is obj:
                    result.append(k)
    return result

20.2 Obtenir le nom de la fonction

20.2.1 Nom de fichier, n° de ligne, nom de fonction sur plusieurs niveaux

import sys
import inspect

def getFrameStr( fr  ):
    _, fileName = os.path.split( fr.f_code.co_filename )
    fname = fr.f_code.co_name
    line = fr.f_lineno
    return "%s(%d).%s" % (fileName,line,  fname)
 
def pfname():
    src1 = ""
    src2 = ""
    src3 = ""    
    nb = len(inspect.stack())
    print( nb )
    if nb > 2 :
        src1 = getFrameStr( sys._getframe(3) )
    if nb > 1 :    
        src2 = getFrameStr( sys._getframe(2) )
    if nb > 0 :    
        src3 = getFrameStr( sys._getframe(1) ) 
    print( "%s %s %s : " %(src1,src2,src3), end="" )

20.2.2 Fonction courante

def pfname( self ):
    print self.__class__.__name__,sys._getframe().f_code.co_name

def func_name():
    return sys._getframe(1).f_code.co_name

20.2.3 Fonction appelante

def pfname( self ):
    print self.__class__.__name__,sys._getframe(1).f_code.co_name

Voir aussi error_in_function

20.2.4 Liens

• www.activestate.com (Recipe 66062: Determining Current Function Name)

21 utilisation des feuilles excel

• Lien vers les modules (source) :

  http://www.python-excel.org/

Working with Excel Files in Python

This site contains pointers to the best information available about working with Excel files in the Python programming language.

Documentation

22 clipboard - presse-papier

22.1 le module clipboard

import clipboard

clipboard.copy("abc")  # now the clipboard content will be string "abc"

text = clipboard.paste()  # text will have the content of clipboard

source  : https://pypi.python.org/pypi/clipboard/0.0.4

il faut aussi : https://pypi.python.org/pypi/pyperclip

22.2 Exemple d'application : copier le chemin du fond d'écran de gnome dans le clipboard

#!/bin/bash
the_bash_file=$0
the_bash_file_dir_name=$(realpath `dirname "$the_bash_file"`)
the_bash_file_base_name=`basename "$the_bash_file"`

imgpath=$(gsettings get org.gnome.desktop.background picture-uri)

python ${the_bash_file_dir_name}/toClipBoard.py "${imgpath}"

#!/bin/python
import os
import sys
import clipboard
clipboard.copy(sys.argv[1][1:-1])

23 Texte

23.1 Supprimer les accents

import unicodedata

def strip_accents(ss):
   s=ss
   return ''.join(c for c in unicodedata.normalize('NFD', s) if unicodedata.category(c) != 'Mn')

23.2 rearrange_text

patWordSep=re.compile("\s+", re.DOTALL )
def rearrange_text( text, tab, rearrange_lf=0 ):
    lines = list()
    if rearrange_lf :
        txt = ''
        ww = text.split('\n')
        if len(ww) == 0 :
            txt = text
        else :
            for w in ww :
                txt += w.strip()+' '
        words = patWordSep.split( txt )
        if len(words) > 0 :
            txt = ""            
            line = ''
            for w in words :
                w=w.strip()
                if len(w) > 0 :
                    if len(line) + len(w) > 80 :
                        lines.append( line )
                        line = w + " "
                    else :
                        line=line+w+" "
            if len( line )>0 :
                lines.append( line )

    else :
        ww = text.split('\n')
        if len(ww) == 0 :
            lines.append( text )
        else :
            for w in ww :
                lines.append( w )

    txt = ''
    for line in lines :
        txt += tab+line+"\n"
    return txt

26 Tkinter : la bibliothèque graphique

• Tkinter:Index

27 Installation

27.1 Installation de librairies

27.1.1 Seulement pour un utilisateur

python setup.py install --user

27.2 Installation avec compilation des modules écrits en C

27.2.1 Sous WINDOWS

Le mieux est d'installer Visual Studio avec l'option « Python tools ... » en choisissant l'option « Installation personnalisée »

27.2.2 Sous Linux

• Exemple : compilation aes pour python 2.5
• alo-aes

28 pylint

pylint --generate-rcfile  > ~/.pylintrc

29 Annotation des functions - typage des arguments

Il est recommandé d'adopter les conventions suivantes pour améliorer la lisibilité du code :

def send_email(address,     # type: Union[str, List[str]]
               sender,      # type: str
               cc,          # type: Optional[List[str]]
               bcc,         # type: Optional[List[str]]
               subject='',
               body=None    # type: List[str]
               ):
    # type: (...) -> bool
    """Send an email message.  Return True if successful."""
    ...

from typing import List

class Example:
    def method(self,
               lst,      # type: List[str]
               opt=0,    # type: int
               *args,    # type: str
               **kwargs  # type: bool
               ):
        # type: (...) -> int
        """Docstring comes after type comment."""
        ...

• source

  http://mypy.readthedocs.io/en/latest/python2.html#multi-line-python-2-function-annotations

• pep-0484

  https://www.python.org/dev/peps/pep-0484

30 PyHelp

• PhHelp

31 Utilisation de python dans le monde du web (World Wide Web)

31.4 Liens

• HOWTO Use Python in the web
• Web Programming in Python

32 Adaptateur Pyhton (wrapper, banding) pour d'autres langages - Bindings - Wrapping

• Python Bindings: Calling C or C++ From Python
• Extending Python with C or C++
• dlsym – A cross-platform symbol locator

33 mp3play

• mp3play

34 debuguer Python

• pylint
• pychecker
• pyflakes
• flake8

  http://www.scipy-lectures.org/advanced/debugging/index.html#debugging-segmentation-faults-using-gdb

35 Différences entre Python 2 et 3

• The key differences between Python 2.7.x and Python 3.x with examples

  http://sebastianraschka.com/Articles/2014_python_2_3_key_diff.html

36 Lines pour débuter

Par ordre de priorité décroissante :

• Pour débuter en Python pour le calcul scientifique :

  Cours CEA, une introduction pour débutant (en principe) : http://iramis.cea.fr/Phocea/file.php?class=cours&file=/olivier.tache/Utilisation_scientifique_de_Python.pdf

• Complément (CEA) pour les graphiques (toujours pour débutants) :

  https://calcul.math.cnrs.fr/attachments/spip/Documents/Ecoles/2010/cours_visu2d_python-v2.pdf

• Cours vidéo en anglais plus général :

  https://open.cs.uwaterloo.ca/python-from-scratch/

• Cours très complet sur le langage Python :

  https://inforef.be/swi/download/apprendre_python3_5.pdf

37 Voir aussi

• PyOpenGL

38 Liens

• ★★ Cours pour débuter en python scientifique

  http://iramis.cea.fr/Phocea/file.php?class=cours&file=/olivier.tache/Utilisation_scientifique_de_Python.pdf

• ctypes
  • Calling C++ Classes from Python, with ctypes…

    http://www.auctoris.co.uk/2017/04/29/calling-c-classes-from-python-with-ctypes/

  • Speeding up element-wise array multiplication in python

    https://stackoverflow.com/questions/19263879/speeding-up-element-wise-array-multiplication-in-python/19458585#19458585

  • f2py import Fortran code in Python

    https://www.scivision.co/f2py-running-fortran-code-in-python-on-windows/

• wrapper C/C++ avec du Python
  • Calling C++ Classes from Python, with ctypes…

    http://www.auctoris.co.uk/2017/04/29/calling-c-classes-from-python-with-ctypes

    https://github.com/Auctoris/ctypes_demo

  • Extending and Embedding the Python 3 Interpreter

    https://docs.python.org/3/extending/index.html

• 10+ Best Python IDES for Software Development (ide Environnement de développement)

  http://hative.com/best-python-ides-for-software-development/

• ★★★ Python From Scratch

  Cours sous forme de videos

  https://open.cs.uwaterloo.ca/python-from-scratch/

• Projet Euler

  https://fr.wikipedia.org/wiki/Project_Euler

  https://projecteuler.net/about

• Computer Science Circles (tutoriel)

  This website teaches computer programming. This skill is very useful: with programming you can automate computer tasks, make art and music, interpret and analyze survey results, build tools for other people, create custom websites, write games, examine genetic data, connect people with each other, and the list goes on and on.

  https://cscircles.cemc.uwaterloo.ca/

• Apprendre à programmer avec Python 3 - Gérard Swinnen

  https://inforef.be/swi/download/apprendre_python3_5.pdf

• How to Think Like a Computer Scientist - thinkpython - Allen Downey, Jerey Elkner, Chris Meyers

  http://www.greenteapress.com/thinkpython/thinkCSpy.pdf

• Domptage de serpent pour les enfants - Apprendre à programmer avec Python - Écrit par Jason R. Briggs et Traduit et adapté par Michel Weinachter

  http://anper95.valdoise.fr/sites/anper/files/document/loisirs/swfk-fr_0.0.9.pdf

• Python for Non-Programmers

  https://wiki.python.org/moin/BeginnersGuide/NonProgrammers

• Repl.it

  Simple, reliable, and portable cloud coding interface. A community of teachers, students and engineers.

  https://repl.it/languages/python

• ANACONDA

  Anaconda is the leading open data science platform powered by Python

  https://www.continuum.io/downloads

• The 10 Most Common Mistakes That Python Developers Make

  https://www.toptal.com/python/top-10-mistakes-that-python-programmers-make

• PEP 3107 -- Function Annotations

http://legacy.python.org/dev/peps/pep-3107/

• urrlib2 opener for SSL proxy (CONNECT method) (Python recipe)
• PyQt
• python doc
• PIL (doc)
• konrad hinsen

http://khinsen.wordpress.com/

http://onclojure.com/2009/03/05/a-monad-tutorial-for-clojure-programmers-part-1/