[Work][Python]Analysis server error log.

Environment: Python 3.8

Key technique: glob, pandas, collections

Task:

Analysis server error log.

Analysis:

When doing reboot, DC, AC cycle test, the system will collect error log from each cycle. Normally, we run at least 100 cycles and takes time to analysis manually. We need to write a script to solve it.

log format

We need output.

  1. fail cycle

Solution:

import glob
import pandas as pd
from collections import Counter
def sel_count(sel):
sel_s=[]
for i in range(0,len(sel)):
sel_s.append(sel[0][i][30:142])
ans=dict(Counter(sel_s))
return ans
file=[]
fail_cycle=[]
p=r"D:\Cycle\**\Summary-Cycle-WcsServer.log"
file.append(glob.glob(p))
log=pd.read_table(file[0][0],header=None,encoding='gb2312',delimiter="\t")
n_cycle=log[0][10][-3:]
targetPattern = r"D:\Cycle\ASEC2\**\**\suspectselentries.decoded.sel.log"
file_d=glob.glob(targetPattern)
file_d2=[]
for i in range(0,int(n_cycle)+1,1):
target="Cycle"+str(i)+"\C"
for j in range(0,len(file_d)):
if(file_d[j].find(target))>0:
file_d2.append(file_d[j])
fail_cycle.append("Cycle"+str(i))
else:
continue
fail_rate="Fail rate is "+str(len(file_d))+"/"+str(n_cycle)
col_sel=[]
all_sel=[]
all_sel.append(fail_rate)
for i in range(0,len(file_d2),1):
sel=pd.read_table(file_d2[i],header=None)
all_sel.append(file_d2[i])
for j in range(0,len(sel),1):
all_sel.append(sel[0][j])
col_sel.append(sel_count(sel))
df=pd.DataFrame(col_sel,fail_cycle)
writer = pd.ExcelWriter('D:\Summary.xlsx', engine='xlsxwriter')
df.to_excel(writer, sheet_name='Sheet1')
df=pd.DataFrame(all_sel)
df.to_excel(writer, sheet_name='Sheet2')
writer.save()

Result:

Interesting in any computer science.