☰

파일에서 특정 필드 추출

tag-icon • tag-icon text-processing awk
파일에서 특정 필드 추출

다음 파일 내용이 있습니다

ALARM: 20190507   1 10:00:09.023000000 10:00:09.023000000 ABB < 50> @160 . "AB12345" 12345 - S . ".." "" "" "POSSIBLE DISK ISSUE (N-to-A) (VAMOS)" "POSSIBLE DISK ISSUE (Y-to-Y) (VAMOS) - At 10:00:09.023, VAR 10 crossed 90 AM at (Alarm number 1213456789). A list issues related to this alarm. See the attached CSV for details (Click 'Export Attachment' at the top of the screen).==20190507.diskissue_g-to-A.12345.Aaslmeer.IssueReferences.1.csv.gz,20190507.diskissue_g-to-A.12345.Aaslmeer.IssueIds.2.csv.gz,20190507.diskissue_g-to-A.12345.Aaslmeer.IssueList.3.csv.gz FIELD*K-ALLOW_PROPRIETARY_VAMOSS-ARR*false*FIELD*K-MEMORY_THRESHOLD-ARR*1*FIELD*K-MEMORY_THRESHOLD-VAL*N/A*FIELD*K-CPU_PERCENT_THRESHOLD-ARR*0.010000000*FIELD*K-CPU_PERCENT_THRESHOLD-VAL*N/A*FIELD*K-VOLUME_LIMIT-ARR*0.000000000*FIELD*K-VOLUME_LIMIT-VAL*N/A*FIELD*K-TIME_THRESHOLD-ARR*00:00:10.000*FIELD*K-TIME_THRESHOLD-VAL*N/A*FIELD*K-PROCESSING_PERCENT_THRESHOLD-ARR*0.000000000*FIELD*K-PROCESSING_PERCENT_THRESHOLD-VAL*N/A*FIELD*K-MATCH_MULTIPLE_TESTS-ARR*false*FIELD*K-EXCLUDE_DIFFERENT_ISSUERS-ARR*false*FIELD*N-CHILD-TYPE*CURRENT*FIELD*N-CHILD-NAME*TRAVEL CURRENT PC04052*FIELD*N-ALARM-TYPE*PRESENT*FIELD*N-OFF-APPLICATION*false*FIELD*N-CTC*27.110000000*FIELD*N-CTC-CU*ABC*FIELD*N-ISSUE-NUM*1240000551*FIELD*N-KNOR-DOWN*N/A*FIELD*N-TRIGGER-TIME-BID*08:27:25.791*FIELD*N-TRIGGER-TIME-ASK*08:27:24.796*FIELD*N-BEN-AVG-DAILY-VOLUME*576525.600000000*FIELD*SYSTEM-TIME*10:02:04.451686000" "" PresentTime

ALARM: 20190507   2 10:00:09.023000000 10:00:09.023000000 ABB < 50> @160 . "LP12345" 12345 - I . ".." "" "" "POSSIBLE DISK ISSUE (Y-to-Y) (CURRENT)" "POSSIBLE DISK ISSUE (Y-to-Y) (VAMOS) - At 10:00:09.023, var 90 crossed  (issue number 12434576589). See the attached CSV for details (Click 'Export Attachment' at the top of the screen).==20190507.diskissue_g-to-A.12345.Aaslmeer.TraderReferences.1.csv.gz,20190507.diskissue_g-to-A.12345.Aaslmeer.IssueIds.2.csv.gz,20190507.diskissue_g-to-A.12345.Aaslmeer.IssueList.3.csv.gz FIELD*K-ALLOW_PROPRIETARY_VAMOSS-ARR*false*FIELD*K-MEMORY_THRESHOLD-ARR*1*FIELD*K-MEMORY_THRESHOLD-VAL*N/A*FIELD*K-CPU_PERCENT_THRESHOLD-ARR*0.010000000*FIELD*K-CPU_PERCENT_THRESHOLD-VAL*N/A*FIELD*K-VOLUME_LIMIT-ARR*0.000000000*FIELD*K-VOLUME_LIMIT-VAL*N/A*FIELD*K-TIME_THRESHOLD-ARR*00:00:10.000*FIELD*K-TIME_THRESHOLD-VAL*N/A*FIELD*K-PRESENT_PERCENT_THRESHOLD-ARR*0.000000000*FIELD*K-PROCESSING_PERCENT_THRESHOLD-VAL*N/A*FIELD*K-MATCH_MULTIPLE_TESTS-ARR*false*FIELD*K-EXCLUDE_DIFFERENT_CASES-ARR*false*FIELD*N-CHILD-TYPE*CURRENT*FIELD*N-CHILD-NAME*TRAVEL Savings PC04052*FIELD*N-ALARM-TYPE*PRESENT*FIELD*N-OFF-APPLICATION*false*FIELD*N-CTC*27.110000000*FIELD*N-CTC-CU*ABC*FIELD*N-ISSUE-NUM*1240000551*FIELD*N-KNOR-DOWN*N/A*FIELD*N-RAISED-TIME-BID*08:27:25.791*FIELD*N-RAISED-TIME-ASK*08:27:24.796*FIELD*N-GUN-AVG-DAILY-VOLUME*576525.600000000*FIELD*SYSTEM-TIME*10:02:04.451686000" "" PresentTime

원하는 출력은 다음과 같습니다

1,K-ALLOW_PROPRIETARY_VAMOSS-ARR,false
1,K-MEMORY_THRESHOLD-ARR,1
1,K-MEMORY_THRESHOLD-VAL,N/A
1,K-CPU_PERCENT_THRESHOLD-ARR,0.010000000
1,K-CPU_PERCENT_THRESHOLD-VAL,N/A
1,K-VOLUME_LIMIT-ARR,0.000000000
1,K-VOLUME_LIMIT-VAL,N/A
1,K-TIME_THRESHOLD-ARR,00:00:10.000
1,K-TIME_THRESHOLD-VAL,N/A
1,K-PROCESSING_PERCENT_THRESHOLD-ARR,0.000000000
1,K-PROCESSING_PERCENT_THRESHOLD-VAL,N/A
1,K-MATCH_MULTIPLE_TESTS-ARR,false
1,K-EXCLUDE_DIFFERENT_ISSUERS-ARR,false
1,N-CHILD-TYPE,CURRENT
1,N-CHILD-NAME,TRAVEL CURRENT PC04052
1,N-ALARM-TYPE,PRESENT
1,N-OFF-APPLICATION,false
1,N-CTC,27.110000000
1,N-CTC-CU,ABC
1,N-ISSUE-NUM,1240000551
1,N-KNOR-DOWN,N/A
1,N-TRIGGER-TIME-BID,08:27:25.791
1,N-TRIGGER-TIME-ASK,08:27:24.796
1,N-BEN-AVG-DAILY-VOLUME,576525.600000000
1,SYSTEM-TIME,10:02:04.451686000
2,K-ALLOW_PROPRIETARY_VAMOSS-ARR,false
2,K-MEMORY_THRESHOLD-ARR,1
2,K-MEMORY_THRESHOLD-VAL,N/A
2,K-CPU_PERCENT_THRESHOLD-ARR,0.010000000
2,K-CPU_PERCENT_THRESHOLD-VAL,N/A
2,K-VOLUME_LIMIT-ARR,0.000000000
2,K-VOLUME_LIMIT-VAL,N/A
2,K-TIME_THRESHOLD-ARR,00:00:10.000
2,K-TIME_THRESHOLD-VAL,N/A
2,K-PRESENT_PERCENT_THRESHOLD-ARR,0.000000000
2,K-PROCESSING_PERCENT_THRESHOLD-VAL,N/A
2,K-MATCH_MULTIPLE_TESTS-ARR,false
2,K-EXCLUDE_DIFFERENT_CASES-ARR,false
2,N-CHILD-TYPE,CURRENT
2,N-CHILD-NAME,TRAVEL Savings PC04052
2,N-ALARM-TYPE,PRESENT
2,N-OFF-APPLICATION,false
2,N-CTC,27.110000000
2,N-CTC-CU,ABC
2,N-ISSUE-NUM,1240000551
2,N-KNOR-DOWN,N/A
2,N-RAISED-TIME-BID,08:27:25.791
2,N-RAISED-TIME-ASK,08:27:24.796
2,N-GUN-AVG-DAILY-VOLUME,576525.600000000
2,SYSTEM-TIME,10:02:04.451686000

다음 명령을 사용하면 필드 3이 없는 각 행의 값을 추출할 수 있습니다. 그러나 원하는 출력을 얻기 위해 각 행에 필드 3을 추가하는 방법을 잘 모르겠습니다.

명령이 시도되었습니다

 zcat input.gz |sed 's/FIELD/\n/g' | grep '^\*' | sed
 's/^*//g;s/*/,/g;s/,$//g;s/" "" PresentTime//g'

수신된 출력

K-ALLOW_PROPRIETARY_VAMOSS-ARR,false
K-MEMORY_THRESHOLD-ARR,1
K-MEMORY_THRESHOLD-VAL,N/A
K-CPU_PERCENT_THRESHOLD-ARR,0.010000000
K-CPU_PERCENT_THRESHOLD-VAL,N/A
K-VOLUME_LIMIT-ARR,0.000000000
K-VOLUME_LIMIT-VAL,N/A
K-TIME_THRESHOLD-ARR,00:00:10.000
K-TIME_THRESHOLD-VAL,N/A
K-PROCESSING_PERCENT_THRESHOLD-ARR,0.000000000
K-PROCESSING_PERCENT_THRESHOLD-VAL,N/A

다른 언어(예: Python)가 사용 가능한 경우에도 관리할 수 있습니다.

참고: 입력 파일은 읽기 전용입니다.

답변1

이 시도,

 while read i; do 
    LineNO=$(echo $i |awk '{print $3}'); 
    echo "$i" |  awk -F 'FIELD' -v a=$LineNO '{for(j=2;j<=NF;j++) print a$j}'| awk -F '[" *]' '{print $1","$2","$3}'; 
 done < input

.gz 파일의 경우

gunzip < input.gz | while read i; do 
    LineNO=$(echo $i |awk '{print $3}'); 
    echo "$i" |  awk -F 'FIELD' -v a=$LineNO '{for(j=2;j<=NF;j++) print a$j}'| awk -F '[" *]' '{print $1","$2","$3}'; 
 done
  • gunzip파일을 읽으십시오.
  • while한 줄씩 읽으십시오.
  • LineNo줄 번호를 저장합니다.
  • FIELD구분 기호로 줄을 나눕니다.
  • for두 번째 열부터 인쇄가 시작됩니다.
  • a줄 번호 인쇄
  • 두 번째 단계에서는 awk쉼표 구분 기호를 사용하여 출력 형식을 지정합니다.

답변2

$ cat tst.awk
BEGIN { OFS="," }
NF {
    idx = $3
    sub(/.* FIELD\*/,"")
    sub(/" .*/,"")
    n = split($0,f,/[*]/)
    for (i=1; i<=n; i+=3) {
        print idx, f[i], f[i+1]
    }
}

.

$ awk -f tst.awk file
1,K-ALLOW_PROPRIETARY_VAMOSS-ARR,false
1,K-MEMORY_THRESHOLD-ARR,1
1,K-MEMORY_THRESHOLD-VAL,N/A
1,K-CPU_PERCENT_THRESHOLD-ARR,0.010000000
1,K-CPU_PERCENT_THRESHOLD-VAL,N/A
1,K-VOLUME_LIMIT-ARR,0.000000000
1,K-VOLUME_LIMIT-VAL,N/A
1,K-TIME_THRESHOLD-ARR,00:00:10.000
1,K-TIME_THRESHOLD-VAL,N/A
1,K-PROCESSING_PERCENT_THRESHOLD-ARR,0.000000000
1,K-PROCESSING_PERCENT_THRESHOLD-VAL,N/A
1,K-MATCH_MULTIPLE_TESTS-ARR,false
1,K-EXCLUDE_DIFFERENT_ISSUERS-ARR,false
1,N-CHILD-TYPE,CURRENT
1,N-CHILD-NAME,TRAVEL CURRENT PC04052
1,N-ALARM-TYPE,PRESENT
1,N-OFF-APPLICATION,false
1,N-CTC,27.110000000
1,N-CTC-CU,ABC
1,N-ISSUE-NUM,1240000551
1,N-KNOR-DOWN,N/A
1,N-TRIGGER-TIME-BID,08:27:25.791
1,N-TRIGGER-TIME-ASK,08:27:24.796
1,N-BEN-AVG-DAILY-VOLUME,576525.600000000
1,SYSTEM-TIME,10:02:04.451686000
2,K-ALLOW_PROPRIETARY_VAMOSS-ARR,false
2,K-MEMORY_THRESHOLD-ARR,1
2,K-MEMORY_THRESHOLD-VAL,N/A
2,K-CPU_PERCENT_THRESHOLD-ARR,0.010000000
2,K-CPU_PERCENT_THRESHOLD-VAL,N/A
2,K-VOLUME_LIMIT-ARR,0.000000000
2,K-VOLUME_LIMIT-VAL,N/A
2,K-TIME_THRESHOLD-ARR,00:00:10.000
2,K-TIME_THRESHOLD-VAL,N/A
2,K-PRESENT_PERCENT_THRESHOLD-ARR,0.000000000
2,K-PROCESSING_PERCENT_THRESHOLD-VAL,N/A
2,K-MATCH_MULTIPLE_TESTS-ARR,false
2,K-EXCLUDE_DIFFERENT_CASES-ARR,false
2,N-CHILD-TYPE,CURRENT
2,N-CHILD-NAME,TRAVEL Savings PC04052
2,N-ALARM-TYPE,PRESENT
2,N-OFF-APPLICATION,false
2,N-CTC,27.110000000
2,N-CTC-CU,ABC
2,N-ISSUE-NUM,1240000551
2,N-KNOR-DOWN,N/A
2,N-RAISED-TIME-BID,08:27:25.791
2,N-RAISED-TIME-ASK,08:27:24.796
2,N-GUN-AVG-DAILY-VOLUME,576525.600000000
2,SYSTEM-TIME,10:02:04.451686000

관련 정보