Pages

Tuesday, July 18, 2017

Python script to scrape score from cricbuzz with desktop alert









Python script to scrape score from cricbuzz with desktop alert


Location :


Requirements




1. Install lxml


    pip install lxml
    pip install lxml==3.6.0


2.Install BeautifulSoup4

    pip install BeautifulSoup4
3.Install pypiwin32 in windows for desktop alert

    pip install pypiwin32

in Linux



    pip install plyer


4. Run the script

























Alert (gnome)


















Friday, June 23, 2017

Python code to monitor conversion process or any process in Linux

Python code to monitor the conversion process 



* Script will monitor the process for specific time after that it will kill it

* it will print the CPU%,Mem%

* First it will get the container PID , then the child PID 


Method Example :



 1


 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
def runOITprocess(timeout):
   commandtouse="cd /opt/app/oit ;./exsimple myfile.docx  output.pdf >  log.txt"  ## conversion process 
   proc=subprocess.Popen(commandtouse, shell=True)
   motherProcess=proc.pid    ## Get Mother process , This process of new shell not real one 
   pidtomonitor="ps -ef | grep "+ str (motherProcess)+" | grep exsimple | grep -v '/bin/sh' | awk '{print $2}'" 
   findingChild = subprocess.Popen(pidtomonitor, shell=True,stdout=subprocess.PIPE)
   childProcess = (findingChild.stdout.read().strip())
   print childProcess  ## This is real process 
   count=0
   awkstring='{print $6" "$9" " $10}'
   fromtopcommand="top -b -n 1 -p "+  (childProcess)+" | tail -1 | awk '"+awkstring+"'"  ## To get Memory%,CPU%
   timedOut=False
   detailsofProces=[]
   while True:
     #print proc.poll()
     if  proc.poll() is not None:
         break
     topcommand  = subprocess.Popen(fromtopcommand, shell=True,stdout=subprocess.PIPE)
     detailsofProces.append(topcommand.stdout.read())
     time.sleep(1)
     stdout.write('\r Running for ' + str (count)  +" s")
     stdout.flush()
     count=count+1
     if timeout==count:
         os.system("kill -9 "+  (childProcess))           ## Killing the child process when time out is over 
         print "Killed due to time out"
         timedOut=True

   CPUL=[]
   MemL=[]
   ResL=[]
   for kk in detailsofProces:
       details=kk.split()
       if details[0].startswith("RES"):
           continue
       ResL.append(details[0])
       CPUL.append(details[1])
       MemL.append(details[2].strip())
   created=False
   if  os.path.isfile("/opt/app/oit/output.pdf"):
       created=True
   distProcessStatus={"timedOut":timedOut,"Time":count,"CPU":CPUL,"Mem":MemL,"Res":ResL,"created":created}
   return distProcessStatus


Example :

[sterin@sterin-linux oittest]$ ./oitUtilis.py
9988
 Running for 7 s{'created': True, 'Mem': ['0.1', '0.2', '0.7', '2.7', '3.1', '3.4', '3.8', '3.8'], 'Res': ['11188', '31964', '107064', '427708', '484072', '536436', '607600', '600400'], 'timedOut': False, 'Time': 8, 'CPU': ['6.7', '13.3', '100.0', '100.0', '100.0', '100.0', '100.0', '100.0']}




Code is highlighted using : http://hilite.me/