The End of 2019

This is the last day of 2019, it was transformative year. I’ve learned a lot by practicing. I faced challenges and learned to make difficult decisions but make the decision rationally. When I faced challenges I like to do binge reading, part of the reasons is that I have the presumption that there must be smart people have been through the challenges I faced today. When I watched the documentary “Inside Bill’s Brain” I noticed that he also has the binge reading habit. He would lock himself in a room and read everything he can about a topic. The books I have finished in the past several weeks are:

  • The man who solved the market
  • A Man of All Markets
  • Dear Chairman
  • Quench your own Thirst
  • The Ride of a Lifetime
  • Never Split the difference
  • My Father’s Business
  • 公司的品格 2
  • 一人の力で日経平均を動かせる男の投資哲学
  • 尋找夢想的家
  • 監工完全上手事件書

I’ve learned to like Audiobooks and I would say it really help a lot on your reading efficiency since it would make you effectively use your time on the road or while you are jogging. This easily adds another hour or two on each of the day. It could also let your eyes to rest, especially for a software engineer where you already use your eyes a lot. I can simply borrow the Audiobook from Singapore library and finish it in one week or two (The deadline of borrowing a book also creates urgency so that you feel like finishing it).

最後、2020年で日本語が上手になりたいです。練習のため、このブログでもとも日本語の文を書きます。真剣に日本語を勉強したい動機は日本の株式の情報収集です。決算報告の日本語分かったら、それが私の競争上の優位性です。後、勉強の仕方を教えます。

Linux materials I read in the past few weeks

I’ve been watching videos and books to get myself familiar with eBPF tooling, more toward Linux network stack and VFS. This post serves as a note on what I’ve learned.

Brendan Gregg has listed the command line tools that are useful for analyzing a Linux instance if he has to ssh into the instance. They could be preliminary analysis to quickly get a feel on what’s going on on a specific instance.

  1. uptime
  2. dmesg -T | tail
  3. vmstat 1
  4. mpstat -P ALL 1
  5. pidstat 1
  6. iostat -xz 1
  7. free -m
  8. sar -n DEV 1
  9. sar -n TCP,ETCP 1
  10. top

To dig deeper into the detail, these eBPF-based tooling would be helpful.

  1. execsnoop
  2. opensnoop
  3. ext4slower
  4. biolatency
  5. biosnoop
  6. cachestat
  7. tcpconnect
  8. tcpaccept
  9. tcpretrans
  10. gethostlatency
  11. runqlat
  12. profile

When the information you are trying to get are not there, and strace is not enough. The following tools could be helpful to peek into the critical paths in the kernel modules.

  1. trace
  2. argdist

A nice book to read is a new book authored by Brendan.

While studying for the SRE materials, I also found Cloudflare has many good blog posts that I could learn from.

A blog post on how to optimize for http2 stack gives really good insight on how the Linux network stack works.

net.core.default_qdisc = fq
net.ipv4.tcp_congestion_control = bbr
net.ipv4.tcp_notsent_lowat = 16384

Their interview questions also makes you start to question yourself if you really know the modern TCP/IP stack.

Archaeology

    What is the lowest TCP port number?

    The TCP frame has an URG pointer field, when is it used?

    Can the RST packet have a payload?

    When is the "flow" field in IPv6 used?

    What does the IP_FREEBIND socket option do?

Forgotten Quirks

    What does the PSH flag actually do?

    The TCP timestamp is implicated in SYN cookies. How?

    Can a "UDP" packet have a checksum field set to zero?

    How does TCP simultaneous open work? Does it actually work?

Fragmentation and Congestion

    What is a stupid window syndrome?

    What are the CWE and ECE flags in TCP header?

    What is the IP ID field and what does it have to do with DF bit? Why do some packets have a non-zero IP ID and a DF set?

Fresh Ideas

    Can a SYN packet have a payload? (hint: new RFC proposals)

    Can a SYN+ACK packet have a payload?

ICMP Path MTU

    ICMP packet-too-big messages are returned by routers and contain a part of the original packet in the payload. What is the minimal length of this payload that is accepted by Linux?

    When an ICMP packet-too-big message is returned by an intermediate router it will have the source IP of that router. In practice though, we often see a source IP of the ICMP message to be identical to the destination IP of the original packet. Why could that happen?

Linux Configuration

    Linux has a "tcp_no_metrics_save" sysctl setting. What does it save and for how long?

    Linux uses two queues to handle incoming TCP connections: the SYN queue and the accept queue. What is the length of the SYN queue?

    What happens if the SYN queue grows too large and overflows?

Touching the router

    What are BGP bogons, and why are they less of a problem now?

    TCP has an extension which adds MD5 checksums to packets. When is it useful?

And finally:

    What are the differences in checksumming algorithms in IPv4 and IPv6?