☰

파일의 특정 줄만 필터링하는 방법은 무엇입니까?

tag-icon • tag-icon awk grep cut json
파일의 특정 줄만 필터링하는 방법은 무엇입니까?

다음 줄이 포함된 파일이 있습니다. 첫 번째 행에 국가 IP 주소를, 두 번째 행에 한 번만, 점수선을 한 번만 보고 싶은데 점수선이 가장 높은 값(이 경우 7.1)이어야 합니다.

{
  "ip": "86.75.227.72",
  "history": [
    {
     "created": "2012-03-22T07:26:00.000Z",
     "reason": "Regional Internet Registry",
     "geo": {
       "country": "France",
       "countrycode": "FR"
     },
     "ip": "86.64.0.0/12",
     "categoryDescriptions": {},
     "reasonDescription": "One of the five RIRs announced a (new) location mapping of the IP.",
     "score": 1,
     "cats": {}
   },
   {
     "created": "2012-04-13T13:34:00.000Z",
     "reason": "DNS heuristics",
     "cats": {
       "Dynamic IPs": 100
     },
     "geo": {
       "country": "France",
       "countrycode": "FR"
     },
     "ip": "86.64.0.0/12",
     "categoryDescriptions": {
       "Dynamic IPs": "This category contains IP addresses of dialup hosts and DSL lines."
     },
     "reasonDescription": "Based on statistical DNS analysis.",
     "score": 1
   },
   {
     "created": "2014-01-22T19:08:00.000Z",
     "reason": "DNS heuristics",
     "cats": {
       "Dynamic IPs": 86
     },
     "geo": {
       "country": "France",
       "countrycode": "FR"
     },
     "ip": "86.72.0.0/14",
     "categoryDescriptions": {
       "Dynamic IPs": "This category contains IP addresses of dialup hosts and DSL lines."
     },
     "reasonDescription": "Based on statistical DNS analysis.",
     "score": 1
   },
   {
     "created": "2014-03-09T13:11:00.000Z",
     "reason": "DNS heuristics",
     "geo": {
       "country": "France",
       "countrycode": "FR"
     },
     "ip": "86.75.224.0/21",
     "cats": {
       "Dynamic IPs": 71
     },
     "categoryDescriptions": {
       "Dynamic IPs": "This category contains IP addresses of dialup hosts and DSL lines."
     },
     "reasonDescription": "Based on statistical DNS analysis.",
     "score": 1
   },
   {
     "created": "2017-07-26T06:24:00.000Z",
     "reason": "Regional Internet Registry",
     "asns": {
       "15557": {
         "Company": "LDCOMNET, FR",
         "cidr": 12
       }
     },
     "geo": {
       "country": "France",
       "countrycode": "FR"
     },
     "ip": "86.75.224.0/21",
     "cats": {
       "Dynamic IPs": 71
     },
     "categoryDescriptions": {
       "Dynamic IPs": "This category contains IP addresses of dialup hosts and DSL lines."
     },
     "reasonDescription": "One of the five RIRs announced a (new) location mapping of the IP.",
     "score": 1
   },
   {
     "created": "2017-10-10T06:23:00.000Z",
     "reason": "Regional Internet Registry",
     "geo": {
       "country": "France",
       "countrycode": "FR"
     },
     "ip": "86.75.224.0/21",
     "cats": {
       "Dynamic IPs": 71
     },
     "categoryDescriptions": {
       "Dynamic IPs": "This category contains IP addresses of dialup hosts and DSL lines."
     },
     "reasonDescription": "One of the five RIRs announced a (new) location mapping of the IP.",
     "score": 1
   },
   {
     "created": "2017-10-18T06:23:00.000Z",
     "reason": "Regional Internet Registry",
     "asns": {
       "15557": {
         "Company": "LDCOMNET, FR",
         "cidr": 12
       }
     },
     "geo": {
       "country": "France",
       "countrycode": "FR"
     },
     "ip": "86.75.224.0/21",
     "cats": {
       "Dynamic IPs": 71
     },
     "categoryDescriptions": {
       "Dynamic IPs": "This category contains IP addresses of dialup hosts and DSL lines."
     },
     "reasonDescription": "One of the five RIRs announced a (new) location mapping of the IP.",
     "score": 1
   },
   {
     "created": "2017-11-20T18:16:00.000Z",
     "reason": "Third party feed",
     "asns": {
       "15557": {
         "Company": "LDCOMNET, FR",
         "cidr": 12
       }
     },
     "geo": {
       "country": "France",
       "countrycode": "FR"
     },
     "ip": "86.75.227.72/32",
     "cats": {
       "Dynamic IPs": 71,
       "Bots": 71
     },
     "categoryDescriptions": {
       "Dynamic IPs": "This category contains IP addresses of dialup hosts and DSL lines.",
       "Bots": "IPs known for botnet-member activity. Devices using these IPs are obviously infected and take part in DDoS-attacks, port-scanning, spam-sending etc."
     },
     "reasonDescription": "This data was imported from a third party feed.",
     "score": 7.1
   },
   {
     "created": "2017-11-25T21:46:00.000Z",
     "reason": "Third party feed",
     "asns": {
       "15557": {
         "Company": "LDCOMNET, FR",
         "cidr": 12
       }
     },
     "geo": {
       "country": "France",
       "countrycode": "FR"
     },
     "ip": "86.75.227.72/32",
     "cats": {
       "Dynamic IPs": 71
     },
     "categoryDescriptions": {
       "Dynamic IPs": "This category contains IP addresses of dialup hosts and DSL lines."
     },
     "reasonDescription": "This data was imported from a third party feed.",
     "score": 1
   }
 ],
 "subnets": [
   {
     "created": "2017-10-18T06:23:00.000Z",
     "reason": "Regional Internet Registry",
     "asns": {
       "15557": {
         "Company": "LDCOMNET, FR",
         "cidr": 12
       }
     },
     "geo": {
       "country": "France",
       "countrycode": "FR"
     },
     "ip": "86.64.0.0",
     "categoryDescriptions": {},
     "reasonDescription": "One of the five RIRs announced a (new) location mapping of the IP.",
     "score": 1,
     "cats": {},
     "subnet": "86.64.0.0/12"
   },
   {
     "created": "2014-03-09T13:11:00.000Z",
     "reason": "DNS heuristics",
     "cats": {
       "Dynamic IPs": 71
     },
     "ip": "86.75.224.0",
     "categoryDescriptions": {
       "Dynamic IPs": "This category contains IP addresses of dialup hosts and DSL lines."
     },
     "reasonDescription": "Based on statistical DNS analysis.",
     "score": 1,
     "subnet": "86.75.224.0/21"
   }
 ],
 "cats": {
   "Dynamic IPs": 71
 },
 "geo": {
   "country": "France",
   "countrycode": "FR"
 },
 "score": 1,
 "reason": "Third party feed",
 "reasonDescription": "This data was imported from a third party feed.",
 "categoryDescriptions": {
   "Dynamic IPs": "This category contains IP addresses of dialup hosts and DSL lines."
 },
 "tags": []
}

"봇": "봇넷 구성원이 활성화한 것으로 알려진 IP입니다. 이러한 IP를 사용하는 장치는 감염되어 DDoS 공격에 참여하는 것으로 보입니다.

"score":7.1}
"geo":{"country":"France"
"score":1}]
"geo":{"country":"France"
"score":1
"score":1
"geo":{"country":"France"
"score":1

답변1

$ jq -r '.history | max_by(.score) | .ip' file.json
86.75.227.72/32

이는 배열에서 최대값을 갖는 항목을 찾는 데 사용됩니다 jq. 일단 발견되면 발견된 항목에서 값이 추출됩니다..history.score.ip

출력 형식은 약간 다릅니다. 여기서는 CSV를 사용하여 IP 주소, 국가, 회사 이름(사용 가능한 경우) 및 점수를 배열로 출력합니다 .history.

$ jq -r '.history[] | [.ip, .geo.country, .asns."15557".Company, .score] | @csv' file.json
"86.64.0.0/12","France",,1
"86.64.0.0/12","France",,1
"86.72.0.0/14","France",,1
"86.75.224.0/21","France",,1
"86.75.224.0/21","France","LDCOMNET, FR",1
"86.75.224.0/21","France",,1
"86.75.224.0/21","France","LDCOMNET, FR",1
"86.75.227.72/32","France","LDCOMNET, FR",7.1
"86.75.227.72/32","France","LDCOMNET, FR",1

이와 동일하지만 가장 높은 점수를 받은 항목만 선택됩니다.

$ jq -r '.history | max_by(.score) | [.ip, .geo.country, .asns."15557".Company, .score] | @csv' file.json
"86.75.227.72/32","France","LDCOMNET, FR",7.1

답변2

떨어져 awk있는:

$ awk '$1=="\"ip\":"{
        ip=$2
       }
       $1=="\"country\":"{
        c[ip]=$2
       }
       $1=="\"score\":" && s[ip]<$2{ 
         s[ip]=$2
       }
       END{
           for(ip in c){
            print ip,c[ip],s[ip]
           }
       }' file 
"86.72.0.0/14", "France", 1
"86.64.0.0/12", "France", 1,
"86.75.224.0/21", "France", 1
"86.75.227.72/32", "France", 7.1
"86.75.227.72", "France", 
"86.75.224.0", "France", 1,

또는 가장 높은 점수의 IP만 원하고 다른 것은 원하지 않는 경우:

$ awk '$1=="\"ip\":"{ip=$2}$1=="\"score\":" && score<$2{score=$2;sip=ip}END{print sip} ' file 
"86.75.227.72/32"

관련 정보