Is the VCR plugged in? Common Sense Troubleshooting For Web Devs

The summers of 6th through 8th grade were incredible. I grew up in North Lake Tahoe and would ride bikes with my brothers and neighborhood kids from daybreak til the streetlights came on. We'd make home videos of us attempting BMX tricks (think Baby-Gap version of the X-Games). We recorded using our family's mini-DV camcorder.

Dad would help connect the camcorder to the VCR and the TV and walk through the labyrinth of VCR remote buttons and menus so that we could transfer the videos to VHS. In the summer of 1997, my dad shared a critically important process with me that I've never forgotten and has been an invaluable framework for troubleshooting ever since. He said, "If the VCR isn't working, you should first check if it's plugged in. Start from the most foundational component (the wall receptacle), then follow the path to the next component in the chain. Ideally, isolating each component and link will ensure all pieces are working as expected. Intuit how the underlying systems work and how they interact."

See, Dad is a genius-level engineer. He ran a commercial refrigeration and appliance business for my entire childhood (which I was lucky enough to learn from). He built houses from the ground up, reincarnated cars, and was a master of all things mechanical, electrical, woodworking, and (to his chagrin) plumbing.

Customers would call and say, "My ice machine isn't making ice." Several times I'd go on jobs with him and watch him use a multimeter to test the voltages at the wall, the switch, the fans, the compressors. Is the electrical all working as expected? What about the water supply? He'd check the inlet valves, pressure, float, and door switches. Is the evaporator working as expected? Are the evaporator coils all frosty, are the condenser coils transferring heat as expected, and so-on. What about the control board? He'd isolate the problem to a single component and replace or repair it. He'd test the system and iterate until it harvested ice as expected.

Once you've isolated a specific component, you can repeat the same process for it itself. Is the component getting power? Is it receiving and/or sending the right signals that are compatible with the other components in the system? Is it damaged?

Tools for Troubleshooting

Dad has a truck full of tools: multimeters, manifold gauges, infrared thermometers (laser guns!). You too, will benefit from collecting tools to help you isolate and identify problems.

In 2011, I spent the year in Afghanistan building tactical networks that ran over fiber optics, satellite, ethernet, and radio.

A lot of my job was configuring Cisco routers and switches. The best troubleshooting tools for that job were ping, traceroute, nslookup, dig and show commands. I'd start by pinging the next hop, the next hop, and the next hop. If the pings were successful, I'd traceroute to the destination to see if the packets followed the expected path. If the packets were not following the expected path, I'd use the show commands to inspect the routing tables and check if RIP, OSPF, and BGP were doing their job correctly. Check the interface status and the logs.

For web development, the best troubleshooting tools are:

1. Logs and Print Statements:

Logs are the most important tool for troubleshooting. Logs are like the black box on an airplane. They record everything that happens in your application. They can tell you what happened, when, and sometimes why it happened. Logs are the first place you should look when something goes wrong. One school of thought is to add tons of print statements to your code to help you understand what's happening. Learn puts p and JSON.pretty_generate in Ruby and the console.log in JavaScript to print out the state of variables and the code flow. Also, use your own logging and learn about log levels to get the right information into your production logs.

2. Debuggers:

Debuggers are tools that allow you to pause the execution of your code and inspect the state of your application. You can set breakpoints in your code and step through it line by line. Debuggers are useful for understanding how your code executes and identifying the bugs' root cause. Some like print statements, and some like debuggers. I like both. I'm kinda old school for debugging ruby, and I like to use byebug. In JavaScript, I like to use debugger or the browser's built-in break statements.

Use step, next, continue, p and puts and l= to print where you are again:

require 'byebug'

def sum(a, b)
  byebug
  a + b
end

def subtract(a, b)
  a - b
end

def multiply(a, b)
  a * b
end

def my_cool_math_thing(a, b)
  byebug
  answer1 = sum(a, b)
  answer1 + subtract(a, b) + multiply(a, b)
end

my_cool_math_thing(1, 2)

3. Browser Developer Tools: Browser developer tools are built into every modern web browser. They allow you to inspect the HTML, CSS, and JavaScript of a web page, as well as monitor network requests, view console logs, and debug JavaScript. Browser developer tools are essential for debugging client-side issues.

4. REPLs: REPL stands for Read-Eval-Print Loop. A REPL is an interactive programming environment that allows you to write and execute code in real time. REPLs are useful for quickly testing code snippets and exploring language features. You can use the console in your browser's developer tools as a client side REPL and your server side framework likely has a built in REPL (e.g. rails console). Using the REPL, you can experiment with bits of code quickly to see how they behave.

5. Automated Tests: Unit tests are automated tests that verify the behavior of a small unit of code in isolation. I like to write unit tests for every bug reported by a user. This way, I can reproduce the bug in a controlled environment and verify that the fix works as expected and that we wont see a regression. There are many different JavaScript test frameworks like Jest, cypress, mocha, and jasmine. We use Rspec and Minitest for unit and integration tests in our rails application.

6. Git Bisect: I've only used this a handful of times in my career, but it can be very handy to identify when a bug was introduced. You start by telling git that the current commit is bad and an earlier commit is good. Git will then checkout a commit in the middle of the range. You test the commit and tell git if it's good or bad. Git will then checkout a commit halfway between the last and current commit. You repeat this process until you find the commit that introduced the bug. It's sort of binary search through history to find a bug.

7. Static Code Analysis: Static code analysis tools analyze your code without executing it. They can identify potential bugs, security vulnerabilities, and code smells. I like to use linters like Rubocop and ESLint to catch syntax errors and enforce code style.

8. Error Messages and Stack Traces: Unfortunately, most error messages look like the villain in a bad action movie. They are scary! Big red letters, often with cryptic language. But, error messages are your best friend. They tell you what went wrong and where it went wrong. They can be cryptic, but they often contain valuable information that can help you identify the root cause of a bug. Stack traces are like a map of the failed code execution path. They show you the sequence of function calls that led to the error. This is your map of all the components that need to be checked -- in order! Usually, I trust that third-party code is working as expected (that's like the wall receptacle, power is probably on, but sometimes you need to check the breaker panel). Do yourself a favor and set up some error monitoring and alerting tools in production so you know when an error happens for a user. We use Sentry at work and I've used Rollbar in the past.

9. Database Management Tools: If you're working with a database, you'll have tools to manage and query the database. Web dev is often CRUD (Create, Read, Update, Delete) data from a database. The database is often the lowest common denominator in the stack. You can use a database management tool to inspect the data in the database, run queries, and check for errors. You can also use the database management tool to check the status of the database server and ensure that it's running as expected. Learn enough SQL to be able to answer some basic questions -- (I still recommend SQL Zoo!). If you're using rails rails dbconsole is a shortcut to log into your database interface and start querying. Recently, I've been wrestling with some Redis memory issues and have found the redis-cli to be invaluable and have learned just enough to be dangerous.

• How many users have an email with test in it?
• What does the weather_days table look like?

Some redis-cli basics, for ya!

10. LLMs and StackOverflow: Learning to ask the right questions of your search engine (Google, GitHub, or StackOverflow) or LLM chat tool like ChatGPT or Claude can help you find solutions faster. The more mature your stack, the more likely there are to be other people who've run into the exact same problem you have and asked about it or created issues on GitHub. As Jon Stuart jokes about prompt engineering, it can pay to become a good "types-question-guy."

Combat Lifesaver Course

In Army basic training, we learned several basic life-saving skills like CPR, applying tourniquets, and treating for shock. In the heat of the moment, it's easy to forget the basics and get lost in the chaos. Many topics in basic training use absurdly simple frameworks so that you can remember them in the heat of the moment. For example, when you find a casualty, you're supposed to remember the ABCs: Airway, Breathing, Circulation.

A. Check their airway, is it clear? If not, clear it.
B. Check their breathing, are they breathing? If not, give them mouth-to-mouth.
C. Check their circulation, do they have a pulse? If not, stop the bleeding (apply a tourniquet) and elevate their legs.

Simple frameworks are easy to remember and can be applied in the heat of the moment. They're also easy to teach and can be applied to various problems. When your site goes down, what are your ABCs?

Side rant: shouldn’t basic life-saving courses be free to the public?! Someone make that happen!

Accelerated Learning Through Troubleshooting

In software engineering, encountering and resolving bugs is not merely an inconvenient part of the development process; it is a critical component that drives rapid skill enhancement and professional growth. I have a theory that the frequency and variety of bugs you encounter directly correlate with the speed at which a software engineer improves. My weaker theory is that this also leads to writing better, more durable code.

As an App Academy instructor, I was lucky enough to accelerate my learning by supporting web development students. While working through the curriculum, they would encounter new bugs and errors that would stump them and other TAs, and I'd get to help find and solve bugs. This experience multiplexed the opportunities for me to encounter new and different (sometimes creative and bewildering) errors and spend all day troubleshooting instead of simply building on my own and only encountering the bugs I created. I learned 6-8 times as much in about 2 years than I had in the previous 5 combined.

Diverse problems required varied approaches and solutions (e.g. comment out half the code), which broadened my problem-solving toolkit.

It deepened my understanding -- I had to understand how different components interact, leading to a more profound grasp of the entire stack.

I learned to anticipate potential issues and write more robust, maintainable code to avoid similar bugs in the future.

The experience built intuition and pattern recognition, helping identify root causes quickly.

For instance, do you want to know the most common root cause for bugs in web applications? SPELLNIG!

Because spelling is one of the most common errors, I prefer to use barewords over @instance_variables in Ruby (thanks Avdi). This way, if I misspell a variable, Ruby will raise an error instead of creating a new variable. Also, a decent reason to use TypeScript over JavaScript and linting tools like ESLint. This is a simple example of how I've learned to write more robust code through troubleshooting.

Tips and Tricks

Walk the path of the request and response - this is the way. Start from the most foundational component (the client) and follow the path to the next component in the chain (the server). Ideally, isolate each component and link to ensure all pieces work as expected. Intuit how the underlying systems work and how they interact with each other.

1. Client: Is the client sending the request as expected? Is the request being sent to the correct endpoint? Is the request being sent with the correct headers and body?

• Make a change to the client code, refresh the page, and verify that you’re changing the code you think you are. (common error is making a change locally but then looking at prod or something).
• Add console log statements to the top of the JavaScript files you expect to be executing, are those printed to the dev console?
• Is the code to execute the request as expected? Use the browser's debugger to find the code that makes the request. Set breakpoints and inspect the request object before it's sent.
• Use the browser's developer tools to inspect the request and response.
• Look in the server logs to see if the request is being received.

2. Server: Is the server receiving the request as expected? Is the server sending the response as expected?

- Look in the server logs to see if the request and the expected URL, headers, and request body are being received.
- Is the expected server code executing?
  - Is the route defined that knows how to pick which code should execute based on the request path?
  - Is the code that should be executing actually executing?

3. Anywhere:

• Follow the control flow, is the execution following the right branches of conditionals?
• Are methods being called with the expected arguments?
• Did the function return the expected value?
• Are the types of objects the expected types?
• Are responses from APIs and Libraries formatted as your code expects?
• Are credentials and constants and environments what you expect?
• Should the message be passed to the class or an instance?
• Is this pointing at what you expect in JavaScript?

Call Your Shots

In pool (like the billiards kind), the badasses call their shots by naming which pocket and which balls will fall before they hit the cue ball. In software engineering, you should say what you expect to happen before you refresh the page or run a test.

If all this sounds obvious, you've been blessed with a good teacher at some point. But, it's also easy to forget when you're in the thick of a problem. It's easy to forget to check if the VCR is plugged in.