[Work][Python]Analysis server error log.
2 min readDec 16, 2020
Environment: Python 3.8
Key technique: glob, pandas, collections
Task:
Analysis server error log.
Analysis:
When doing reboot, DC, AC cycle test, the system will collect error log from each cycle. Normally, we run at least 100 cycles and takes time to analysis manually. We need to write a script to solve it.
log format
We need output.
- fail cycle
- error sel
- 3. same error sel number in each cycle 4.
- fail rate
- Raw sel
Solution:
import glob
import pandas as pd
from collections import Counterdef sel_count(sel):
sel_s=[]
for i in range(0,len(sel)):
sel_s.append(sel[0][i][30:142])
ans=dict(Counter(sel_s))
return ansfile=[]
fail_cycle=[]
p=r"D:\Cycle\**\Summary-Cycle-WcsServer.log"
file.append(glob.glob(p))
log=pd.read_table(file[0][0],header=None,encoding='gb2312',delimiter="\t")
n_cycle=log[0][10][-3:]targetPattern = r"D:\Cycle\ASEC2\**\**\suspectselentries.decoded.sel.log"
file_d=glob.glob(targetPattern)
file_d2=[]
for i in range(0,int(n_cycle)+1,1):
target="Cycle"+str(i)+"\C"
for j in range(0,len(file_d)):
if(file_d[j].find(target))>0:
file_d2.append(file_d[j])
fail_cycle.append("Cycle"+str(i))
else:
continuefail_rate="Fail rate is "+str(len(file_d))+"/"+str(n_cycle)
col_sel=[]
all_sel=[]
all_sel.append(fail_rate)
for i in range(0,len(file_d2),1):
sel=pd.read_table(file_d2[i],header=None)
all_sel.append(file_d2[i])
for j in range(0,len(sel),1):
all_sel.append(sel[0][j])
col_sel.append(sel_count(sel))df=pd.DataFrame(col_sel,fail_cycle)
writer = pd.ExcelWriter('D:\Summary.xlsx', engine='xlsxwriter')
df.to_excel(writer, sheet_name='Sheet1')
df=pd.DataFrame(all_sel)
df.to_excel(writer, sheet_name='Sheet2')
writer.save()
Result:
