I was fortunate enough to be selected to participate in the ARIN51 fellowship program. Attending the event in Tampa gave me a glimpse of how the policies discussed shape the broader internet. I highly recommend anyone in this field to consider participating in the ARIN community and apply for the fellowship. ARIN 51 discussed a number of draft policy recommendations , service improvements , and community grants initiatives.

Secure Routing
One the key themes of ARIN 51 in my opinion was RPKI. Securing routing is clearly of the most important initiatives. The ROA-Thon hosted by Brad Gorman , The Draft Policy Recommendations ARIN-2021-8 , The Grants Presentation from the DNS Research Federation on the state of RPKI.

Draft Policy Recommendations ARIN-2021-8
This was a Draft Policy Recommendation to remove the ‘Autonomous System Originations’ Field. The OriginAS field has real world use case for organizations that do not use Internet Routing Registries which groups Public Prefixes — ASN under an object name . While most organization are likley using IRR , With removing the OrginAS field and the retirement of the legacy unauthorized ARIN-NONAUTH IRR. The community is driven to making routing more secure and embracing solutions that ensure more accurate , cryptographic secure records.

ROA-THON
The ROA-THON hosted by Brad Gorman, where explained the technical details , benefits and limits of how RPKI can create more secure routing and how the hosted ARIN RPKI product functions today and new features such as certificates never expiring. The key take away from the ROA-THON , lunch tabletop topics and presentation is to at LEAST sign the ROA for the resources, even if your organization is not ready to make decision with RPKI , Signing the routes only benefits and protects the resource holder.

Tech Brief of RPKI

• I recommend reviewing `https://rpki.readthedocs.io/en/latest/` in details for specifics.
• -Great tool `https://irrexplorer.nlnog.net/`
• Resource Public Key Infrastructure – RPKI
• – Route Origin Authorization – ROA
• – Allows Holders of Internet Number Resources to make signed statements about them
• – Third Party resources can download these statements as x509 Certificates
• – These signed certificates assure that a person who administers the Number Resources. Limits attack Surface
• – Does not address Path Validation
• – ROAs allow holders to secure resources so ISPs that make decisions based on RPKI can ensure validity of the ROA
• – Protects against BGP Hijacking
• – Protects against human error , ROA have prefix lengths that can be used to ensure correct advertisements are expected

DNS Research Federation
ARIN provides grants to organization that aim to improve the overall internet. One of the grant recipients was the DNS Research Federation. https://dnsrf.org/. DNSRF was given a grant to evaluate the state of deployment of RPKI in the ARIN region and encourage greater scrutiny over routing and security . The project analyzed the number of RPKIs in use ARIN for ipv4 , ipv6 versus the global deployment. The ARIN region has about 25% of IPv4 covered under RPKLI and ~50% for ipv6.


Number Resource Organization / Address Supporting Organization

One the important items to remember is ARIN is one the five RIRs but works with ICANN. The NRO is the group within that that is part of the ASO group inside of ICANN (confusing yes) but the important item is each RIR allocates members to ASO to work collaboratively on global policy. Kevin Blumberg gave an update presentation on the ASO AC Update. The idea here is that each RIR has to continuously work with the global community on evolving policies.

IPV6
Since ARIN is one the five RIR responsible for administering policy for Numbered Resources , it is in the best interest to move away from the scarcity limitation that ipv4 address space is facing. During ARIN 51 there was a number of success stories of IPV6 deployments including for speakers from Virgnia Tech , AWS and Telus. All of these institutions deployed ipv6 into their networks , Virginia Tech did it 25 years ago !

The mobile carriers of the world are IPV6 .The point is that ipv6 is a real , robust protocol that is becoming superior to *legacy ip*. Albeit some challenges with IPv6 are in the application development space , if an organization treats ipv6 as a requirement
wants to *simplifying* their network (GOODBYE NAT!) . IPv6 is an excellent solution that is not that difficult to deploy considering.

Number Resource Policy Manual & IPv6
One of the more interesting items I discovered is that ARIN in order to facilitate the adoption of IPV6 makes special use cases of IPv4 including micro-allocation to support network infrastructure. Under NRPM section 4.4 this allocation includes space for Internet Peering Exchange points. I found it interesting since this is services like de-cix which I realized when reading the IP Assignments. NRPM Section 4.10 is also another interesting item , under this policy , ARIN set aside a /10 of ipv4 space to promote ipv6
deployments. An applicant must demonstrate an immediate need for a dual stack deployment and never received resources under the policy previously nor have other allocations that will satisfy the need.
– `https://www.arin.net/reference/research/statistics/microallocations/`

Vector is an observability data pipeline that collects , transforms , and routes logs , metrics to any vendor or destination. Vector is open source, cloud native and is the best performing tool in the market.
The various components of Vector allow it to have multiple deployment strategies . Vector can be deployed as an agent and/or aggregator into multiple topologies [ Distributed , Centralized and Stream ] .
Vector components are Sources , Transforms and Sinks.

Vector Source
A source defines where data is ingested from. A source defines where data is pulled from. There is an extensive list of sources including examples AWS , Docker Logs , Server Logs , Server Exec , SNMP , SYSLOG , Apache and Vector.

sources:
  metrics:
    type: host_metrics
    collectors:
      - cpu
      - disk
      - filesystem
      - load
      - host
      - memory
      - network
    scrape_interval_secs: 15
  
 my_source_id:
    type: exec

Vector Transforms
Transform literally transform the raw data that are transported . This can be parsing , filtering , sampling or tagging. You can transform the data in the pipeline however you see fit. Example from the config.yaml file

transforms:
  get_error_log:
    type: filter
    inputs:
      - vector_error
    condition: |
      contains(string!(.message) , "error")

Vector Sinks
At the end of the pipeline , the information needs to be sent somewhere. The Sink is a destination for the events. The example below will send the errors from the transform set above to a log file. This is a simple implementation and data can be sent to Prometheus , AWS S3 Bucket , Kafka , etc.

sinks:
  errors_to_file:
    type: file
    inputs:
      - get_error_log
    compression: none
    encoding:
      codec: json
    path: /tmp/vector-error-%Y-%m-%d.log

Using Vector in combination with Prometheus / Grafana allows for power distributed monitoring systems.

Links:
https://vector.dev/

If you enjoy , please feel free to subscribe.

Assuming a new role has allowed me to learn more about Public cloud platforms . Specifically AWS , TIL there is a AWS CLI that can be used to interact with the services for the organization.

Install AWSCLI

### CLI Download for MACOS:

curl "https://awscli.amazonaws.com/AWSCLIV2.pkg" -o "AWSCLIV2.pkg"
sudo installer -pkg AWSCLIV2.pkg -target /

### AWSCLIV2.pkg is the name of the file saved on the local machine 

### Install the Package 
sudo installer -pkg ./AWSCLIV2.pkg -target /

### Confirm Installation 
user@ip-172-16-0-179 vector--demo % which aws
/usr/local/bin/aws

### Add directory to $PATH
export PATH=$PATH:/usr/local/bin/aws

### Confirm 
user@ip-172-16-0-179 ~ % echo $PATH
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin

Links:
https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html#getting-started-install-instructions

In the middle of July of 2022, I was scrolling though my gmail account. Reviewing NANOG emails distro when John Curran shared an opportunity to get involved with the community . The fellowship is an opportunity to receive an in depth mentored introduction to ARIN.

ARIN is the American Registry for Internet Numbers. The focus of ARIN is the efficient allocation of resources, educate the community and enhance their services.

Having recently attended an open session for RPKI from ARIN , having a role currently managing ARIN resources, I am eager to learn direct from community member professionals.

Participating in the community has been a personal goal of mine for several years and this fellowship also allows to me to attend virtually ARIN/NANOG 50. This seems to be an opportunity to understand the governance

I am hoping to gain a greater understanding of how to be be more effective with the resource I am responsible for , implement RPKI , and be a more active member of the community at large.

https://www.arin.net/announcements/20220916/

Brief Backstory

What attracted me into technology is the problem solving aspect of it. It’s a personal driver that has sustained me in my profession.
Using the proper tools is a great way to help solve a mystery. In this particular instance , my colleague and I were reviewing something simple … The network was not learning any Layer 2 Addressing from the new storage boxes trunked over.
Seems like a really easy item to configure right?
Since the links did show multicast traffic across it and Refusing to give us , I decided to revisit Ethanalyzer

Technical Description

Ethanalyzer is the NX-OS flavor of Tshark , which is the terminal version of Wireshark. Ethanalyzer can capture inband and management traffic and filter packets. What is interesting is you can use both tcpdump or wireshark capture syntax .

Hurry up ! I need to troubleshoot !

Yes , I wrote this because I could not figure out how to filter results quickly.
Filtering packets is where I got stopped briefly because there is

• display-filter
  • display-filter == wireshark
• capture-filter
  • capture-filter == tcpdump

Wireshark Display Filter

Wireshark / Display Filter 

Filter by Port

nxos# ethanalyzer local interface inband display-filter "tcp.port == 179" limit-captured-frames 400
Capturing on inband
2022-08-13 12:44:42.153103    10.1.1.53 -> 10.1.1.52    BGP KEEPALIVE Message
2022-08-13 12:44:42.153116    10.1.2.53 -> 10.1.2.52    BGP KEEPALIVE Message


Filter by MAC Address 
```ethanalyzer local interface inband display-filter "eth.addr eq 00:50:56:01:0a:a8" ```

Filter by Vlan ID 
``` ethanalyzer local interface inband display-filter "vlan.id eq 3902" limit-captured-frames 400```

- Filter by IP Address
``` ethanalyzer local interface inband display-filter "ip.addr eq 192.168.21.160" limit-captured-frames 400```

TCPDUMP Capture Filter

TCPDUMP / Display Filter 
nxos# ethanalyzer local interface inband capture-filter "tcp port 179" 
Capturing on inband
2022-08-13 12:46:13.741855    10.1.3.52 -> 10.1.3.53    BGP KEEPALIVE Message
2022-08-13 12:46:13.741916  10.200.0.15 -> 10.200.200.27 BGP KEEPALIVE Message

Capture Output to a file

The display at the end will output the content while capturing

ethanalyzer local interface inband write bootflash:///20220813.pcap display 
Capturing on inband
2022-08-13 13:22:27.280908    10.1.1.52 -> 10.1.1.53    BGP KEEPALIVE Message
2022-08-13 13:22:27.280972    10.1.1.53 -> 10.1.1.52    TCP bgp > 55311 [ACK] Seq=1 Ack=20 Win=7280 Len=0 TSV=931077017 TSER=2303421976

Read the output of the pcap

nxos# ethanalyzer local read bootflash:///20220813.pcap
2022-08-13 13:22:27.280908    10.1.1.52 -> 10.1.1.53    BGP KEEPALIVE Message
2022-08-13 13:22:27.280972    10.1.1.53 -> 10.1.1.52    TCP bgp > 55311 [ACK] Seq=1 Ack=20 Win=7280 Len=0 TSV=931077017 TSER=2303421976
2022-08-13 13:22:27.280986  10.200.0.13 -> 10.200.200.27 BGP KEEPALIVE Message
2022-08-13 13:22:27.281002 10.200.200.27 -> 10.200.0.13  TCP 39174 > bgp [ACK] Seq=1 Ack=20 Win=30477 Len=0 TSV=931077017 TSER=2303421976
2022-08-13 13:22:27.343513 00:35:1a:9d:ab:ef -> 01:80:c2:00:00:00 STP RST. Root = 32768/1/00:23:04:ee:be:64  Cost = 0  Port = 0x9063
2022-08-13 13:22:27.343566 00:35:1a:9d:ab:ef -> 01:00:0c:cc:cc:cd STP RST. Root = 32768/1/00:23:04:ee:be:64  Cost = 0  Port = 0x9063
2022-08-13 13:22:27.343601 00:35:1a:9d:ab:ef -> 01:00:0c:cc:cc:cd STP RST. Root = 32768/745/00:23:04:ee:be:64  Cost = 0  Port = 0x9063
2022-08-13 13:22:27.674072    10.1.3.52 -> 10.1.3.53    BGP KEEPALIVE Message
2022-08-13 13:22:27.674138    10.1.3.53 -> 10.1.3.52    TCP bgp > 33813 [ACK] Seq=1 Ack=20 Win=7280 Len=0 TSV=931077135 TSER=2303326167
2022-08-13 13:22:27.674151  10.200.0.15 -> 10.200.200.27 BGP KEEPALIVE Message

Ultimately

The root cause of the issue ended up being a misconfigured route map which prevented traffic between the overlays. While Ethanalyzer did not directly solve the problem , where I found this helpful was in using to troubleshoot connectivity in my new overlay networks.
In my previous experience with Ethanalyzer , I was reluctant to use it because I did not know to filter. Even in one of the Cisco press books .

While using Ethanalyzer, specifying the filters is easier for someone who is familiar with Wireshark filters. The syntax for both the capture filter and the display filter is different. Table 2-1 lists some of the common filters and their syntax with the **capture-filter** and **display-filter** options.

While the author does explain great technical material its the above paragraph that I wish contained the terse point of display-filter == wireshark , capture-filter == tcpdump.

I want to thank Ivan Shirshin who wrote a community blog post in 2013 that made this information clear as day to me. This allowed me to use Ethanalyzer effectively.

Citing my sources / links

• Wireshark Display Filters Man Page
  • https://www.wireshark.org/docs/man-pages/wireshark-filter.html
• Cisco Community Post from Ivan Shirshin
  • https://community.cisco.com/t5/networking-knowledge-base/using-ethanalyzer-on-nexus-platform-for-control-plane-and-data/ta-p/3142665
• TCPDUMP Man Page
  • https://www.tcpdump.org/manpages/tcpdump.1.html
• Cisco NX-OS Troubleshooting
  • https://www.cisco.com/c/en/us/td/docs/switches/datacenter/nexus9000/sw/6-x/troubleshooting/guide/b_Cisco_Nexus_9000_Series_NX-OS_Troubleshooting_Guide/b_Cisco_Standalone_Series_NX-OS_Troubleshooting_Guide_chapter_010000.html#reference_EF208AE32A30415F8F172A5E417868A8
• Cisco Press Books
  • https://www.ciscopress.com/articles/article.asp?p=2928194&seqNum=2