This is an extract from GitHub - jwasham/coding-interview-university: A complete computer science study plan to become a software engineer. (opens in a new tab)

Toggling the checklist has not been implemented so I would suggest cloning the original repo and using some markdown editor like Vscode or obsidian to toggle the checkboxes.

Coding Interview University

• Algorithmic complexity / Big-O / Asymptotic analysis
• Data Structures
• More Knowledge
• Trees
• Sorting
• Graphs
• Even More Knowledge
• Final Review
• Update Your Resume
• Interview Process & General Interview Prep
• Be thinking of for when the interview comes
• Have questions for the interviewer
• Once You've Got The Job
• Additional Books
• System Design, Scalability, Data Handling
• Additional Learning
• Additional Detail on Some Subjects
• Video Series
• Computer Science Courses
• Algorithms implementation
• Papers
• LICENSE

Algorithmic complexity / Big-O / Asymptotic analysis

• Nothing to implement here, you're just watching videos and taking notes! Yay!
• There are a lot of videos here. Just watch enough until you understand it. You can always come back and review.
• Don't worry if you don't understand all the math behind it.
• You just need to understand how to express the complexity of an algorithm in terms of Big-O.
• Harvard CS50 - Asymptotic Notation (video) (opens in a new tab)
• Big O Notations (general quick tutorial) (video) (opens in a new tab)
• Big O Notation (and Omega and Theta) - best mathematical explanation (video) (opens in a new tab)
• Skiena (video) (opens in a new tab)
• UC Berkeley Big O (video) (opens in a new tab)
• Amortized Analysis (video) (opens in a new tab)
• TopCoder (includes recurrence relations and master theorem):
  • Computational Complexity: Section 1 (opens in a new tab)
  • Computational Complexity: Section 2 (opens in a new tab)
• Cheat sheet (opens in a new tab)
• [Review] Analyzing Algorithms (playlist) in 18 minutes (video) (opens in a new tab)

Well, that's about enough of that.

When you go through "Cracking the Coding Interview", there is a chapter on this, and at the end there is a quiz to see if you can identify the runtime complexity of different algorithms. It's a super review and test.

Data Structures

• Arrays

  • About Arrays:
    • Arrays CS50 Harvard University (opens in a new tab)
    • Arrays (video) (opens in a new tab)
    • UC Berkeley CS61B - Linear and Multi-Dim Arrays (video) (opens in a new tab) (Start watching from 15m 32s)
    • Dynamic Arrays (video) (opens in a new tab)
    • Jagged Arrays (video) (opens in a new tab)
  • Implement a vector (mutable array with automatic resizing):
    • Practice coding using arrays and pointers, and pointer math to jump to an index instead of using indexing.
    • New raw data array with allocated memory
      • can allocate int array under the hood, just not use its features
      • start with 16, or if the starting number is greater, use power of 2 - 16, 32, 64, 128
    • size() - number of items
    • capacity() - number of items it can hold
    • is_empty()
    • at(index) - returns the item at a given index, blows up if index out of bounds
    • push(item)
    • insert(index, item) - inserts item at index, shifts that index's value and trailing elements to the right
    • prepend(item) - can use insert above at index 0
    • pop() - remove from end, return value
    • delete(index) - delete item at index, shifting all trailing elements left
    • remove(item) - looks for value and removes index holding it (even if in multiple places)
    • find(item) - looks for value and returns first index with that value, -1 if not found
    • resize(new_capacity) // private function
      • when you reach capacity, resize to double the size
      • when popping an item, if the size is 1/4 of capacity, resize to half
  • Time
    • O(1) to add/remove at end (amortized for allocations for more space), index, or update
    • O(n) to insert/remove elsewhere
  • Space
    • contiguous in memory, so proximity helps performance
    • space needed = (array capacity, which is >= n) * size of item, but even if 2n, still O(n)

• Linked Lists

  • Description:
    • Linked Lists CS50 Harvard University (opens in a new tab) - this builds the intuition.
    • Singly Linked Lists (video) (opens in a new tab)
    • CS 61B - Linked Lists 1 (video) (opens in a new tab)
    • CS 61B - Linked Lists 2 (video) (opens in a new tab)
    • [Review] Linked lists in 4 minutes (video) (opens in a new tab)
  • C Code (video) (opens in a new tab)
    • not the whole video, just portions about Node struct and memory allocation
  • Linked List vs Arrays:
    • Core Linked Lists Vs Arrays (video) (opens in a new tab)
    • In The Real World Linked Lists Vs Arrays (video) (opens in a new tab)
  • Why you should avoid linked lists (video) (opens in a new tab)
  • Gotcha: you need pointer to pointer knowledge: (for when you pass a pointer to a function that may change the address where that pointer points) This page is just to get a grasp on ptr to ptr. I don't recommend this list traversal style. Readability and maintainability suffer due to cleverness.
    • Pointers to Pointers (opens in a new tab)
  • Implement (I did with tail pointer & without):
    • size() - returns the number of data elements in the list
    • empty() - bool returns true if empty
    • value_at(index) - returns the value of the nth item (starting at 0 for first)
    • push_front(value) - adds an item to the front of the list
    • pop_front() - remove the front item and return its value
    • push_back(value) - adds an item at the end
    • pop_back() - removes end item and returns its value
    • front() - get the value of the front item
    • back() - get the value of the end item
    • insert(index, value) - insert value at index, so the current item at that index is pointed to by the new item at the index
    • erase(index) - removes node at given index
    • value_n_from_end(n) - returns the value of the node at the nth position from the end of the list
    • reverse() - reverses the list
    • remove_value(value) - removes the first item in the list with this value
  • Doubly-linked List
    • Description (video) (opens in a new tab)
    • No need to implement

• Stack

  • Stacks (video) (opens in a new tab)
  • [Review] Stacks in 3 minutes (video) (opens in a new tab)
  • Will not implement. Implementing with the array is trivial

• Queue

  • Queue (video) (opens in a new tab)
  • Circular buffer/FIFO (opens in a new tab)
  • [Review] Queues in 3 minutes (video) (opens in a new tab)
  • Implement using linked-list, with tail pointer:
    • enqueue(value) - adds value at a position at the tail
    • dequeue() - returns value and removes least recently added element (front)
    • empty()
  • Implement using a fixed-sized array:
    • enqueue(value) - adds item at end of available storage
    • dequeue() - returns value and removes least recently added element
    • empty()
    • full()
  • Cost:
    • a bad implementation using a linked list where you enqueue at the head and dequeue at the tail would be O(n) because you'd need the next to last element, causing a full traversal of each dequeue
    • enqueue: O(1) (amortized, linked list and array [probing])
    • dequeue: O(1) (linked list and array)
    • empty: O(1) (linked list and array)

• Hash table

  • Videos:

    • Hashing with Chaining (video) (opens in a new tab)
    • Table Doubling, Karp-Rabin (video) (opens in a new tab)
    • Open Addressing, Cryptographic Hashing (video) (opens in a new tab)
    • PyCon 2010: The Mighty Dictionary (video) (opens in a new tab)
    • PyCon 2017: The Dictionary Even Mightier (video) (opens in a new tab)
    • (Advanced) Randomization: Universal & Perfect Hashing (video) (opens in a new tab)
    • (Advanced) Perfect hashing (video) (opens in a new tab)
    • [Review] Hash tables in 4 minutes (video) (opens in a new tab)

  • Online Courses:

    • Core Hash Tables (video) (opens in a new tab)
    • Data Structures (video) (opens in a new tab)
    • Phone Book Problem (video) (opens in a new tab)
    • distributed hash tables:
      • Instant Uploads And Storage Optimization In Dropbox (video) (opens in a new tab)
      • Distributed Hash Tables (video) (opens in a new tab)

  • Implement with array using linear probing

    • hash(k, m) - m is the size of the hash table
    • add(key, value) - if the key already exists, update value
    • exists(key)
    • get(key)
    • remove(key)

More Knowledge

• Binary search

  • Binary Search (video) (opens in a new tab)
  • Binary Search (video) (opens in a new tab)
  • detail (opens in a new tab)
  • blueprint (opens in a new tab)
  • [Review] Binary search in 4 minutes (video) (opens in a new tab)
  • Implement:
    • binary search (on a sorted array of integers)
    • binary search using recursion

• Bitwise operations

  • Bits cheat sheet (opens in a new tab)
    • you should know many of the powers of 2 from (2^1 to 2^16 and 2^32)
  • Get a really good understanding of manipulating bits
    • words (opens in a new tab)
    • Good intro: Bit Manipulation (video) (opens in a new tab)
    • C Programming Tutorial 2-10: Bitwise Operators (video) (opens in a new tab)
    • Bit Manipulation (opens in a new tab)
    • Bitwise Operation (opens in a new tab)
    • Bithacks (opens in a new tab)
    • The Bit Twiddler (opens in a new tab)
    • The Bit Twiddler Interactive (opens in a new tab)
    • Bit Hacks (video) (opens in a new tab)
    • Practice Operations (opens in a new tab)
  • 2s and 1s complement
    • Binary: Plusses & Minuses (Why We Use Two's Complement) (video) (opens in a new tab)
    • 1s Complement (opens in a new tab)
    • 2s Complement (opens in a new tab)
  • Count set bits
    • 4 ways to count bits in a byte (video) (opens in a new tab)
    • Count Bits (opens in a new tab)
    • How To Count The Number Of Set Bits In a 32 Bit Integer (opens in a new tab)
  • Swap values:
    • Swap (opens in a new tab)
  • Absolute value:
    • Absolute Integer (opens in a new tab)

Trees

• Trees - Intro

  • Intro to Trees (video) (opens in a new tab)
  • Tree Traversal (video) (opens in a new tab)
  • BFS(breadth-first search) and DFS(depth-first search) (video) (opens in a new tab)
    • BFS notes:
      • level order (BFS, using queue)
      • time complexity: O(n)
      • space complexity: best: O(1), worst: O(n/2)=O(n)
    • DFS notes:
      • time complexity: O(n)
      • space complexity: best: O(log n) - avg. height of tree worst: O(n)
      • inorder (DFS: left, self, right)
      • postorder (DFS: left, right, self)
      • preorder (DFS: self, left, right)
  • [Review] Breadth-first search in 4 minutes (video) (opens in a new tab)
  • [Review] Depth-first search in 4 minutes (video) (opens in a new tab)
  • [Review] Tree Traversal (playlist) in 11 minutes (video) (opens in a new tab)

• Binary search trees: BSTs

  • Binary Search Tree Review (video) (opens in a new tab)
  • Introduction (video) (opens in a new tab)
  • MIT (video) (opens in a new tab)
  • C/C++:
    • Binary search tree - Implementation in C/C++ (video) (opens in a new tab)
    • BST implementation - memory allocation in stack and heap (video) (opens in a new tab)
    • Find min and max element in a binary search tree (video) (opens in a new tab)
    • Find the height of a binary tree (video) (opens in a new tab)
    • Binary tree traversal - breadth-first and depth-first strategies (video) (opens in a new tab)
    • Binary tree: Level Order Traversal (video) (opens in a new tab)
    • Binary tree traversal: Preorder, Inorder, Postorder (video) (opens in a new tab)
    • Check if a binary tree is a binary search tree or not (video) (opens in a new tab)
    • Delete a node from Binary Search Tree (video) (opens in a new tab)
    • Inorder Successor in a binary search tree (video) (opens in a new tab)
  • Implement:
    • insert // insert value into tree (opens in a new tab)
    • get_node_count // get count of values stored
    • print_values // prints the values in the tree, from min to max
    • delete_tree
    • is_in_tree // returns true if a given value exists in the tree
    • get_height // returns the height in nodes (single node's height is 1) (opens in a new tab)
    • get_min // returns the minimum value stored in the tree
    • get_max // returns the maximum value stored in the tree
    • is_binary_search_tree (opens in a new tab)
    • delete_value
    • get_successor // returns the next-highest value in the tree after given value, -1 if none

• Heap / Priority Queue / Binary Heap

  • visualized as a tree, but is usually linear in storage (array, linked list)
  • Heap (opens in a new tab)
  • Introduction (video) (opens in a new tab)
  • Binary Trees (video) (opens in a new tab)
  • Tree Height Remark (video) (opens in a new tab)
  • Basic Operations (video) (opens in a new tab)
  • Complete Binary Trees (video) (opens in a new tab)
  • Pseudocode (video) (opens in a new tab)
  • Heap Sort - jumps to start (video) (opens in a new tab)
  • Heap Sort (video) (opens in a new tab)
  • Building a heap (video) (opens in a new tab)
  • MIT 6.006 Introduction to Algorithms: Binary Heaps (opens in a new tab)
  • CS 61B Lecture 24: Priority Queues (video) (opens in a new tab)
  • Linear Time BuildHeap (max-heap) (opens in a new tab)
  • [Review] Heap (playlist) in 13 minutes (video) (opens in a new tab)
  • Implement a max-heap:
    • insert
    • sift_up - needed for insert
    • get_max - returns the max item, without removing it
    • get_size() - return number of elements stored
    • is_empty() - returns true if the heap contains no elements
    • extract_max - returns the max item, removing it
    • sift_down - needed for extract_max
    • remove(x) - removes item at index x
    • heapify - create a heap from an array of elements, needed for heap_sort
    • heap_sort() - take an unsorted array and turn it into a sorted array in place using a max heap or min heap

Sorting

• Notes:

  • Implement sorts & know best case/worst case, average complexity of each:
    • no bubble sort - it's terrible - O(n^2), except when n <= 16
  • Stability in sorting algorithms ("Is Quicksort stable?")
    • Sorting Algorithm Stability (opens in a new tab)
    • Stability In Sorting Algorithms (opens in a new tab)
    • Stability In Sorting Algorithms (opens in a new tab)
    • Sorting Algorithms - Stability (opens in a new tab)
  • Which algorithms can be used on linked lists? Which on arrays? Which of both?
    • I wouldn't recommend sorting a linked list, but merge sort is doable.
    • Merge Sort For Linked List (opens in a new tab)

• For heapsort, see the Heap data structure above. Heap sort is great, but not stable

• Sedgewick - Mergesort (5 videos) (opens in a new tab)

  • 1. Mergesort (opens in a new tab)
  • 2. Bottom-up Mergesort (opens in a new tab)
  • 3. Sorting Complexity (opens in a new tab)
  • 4. Comparators (opens in a new tab)
  • 5. Stability (opens in a new tab)

• Sedgewick - Quicksort (4 videos) (opens in a new tab)

  • 1. Quicksort (opens in a new tab)
  • 2. Selection (opens in a new tab)
  • 3. Duplicate Keys (opens in a new tab)
  • 4. System Sorts (opens in a new tab)

• UC Berkeley:

  • CS 61B Lecture 29: Sorting I (video) (opens in a new tab)
  • CS 61B Lecture 30: Sorting II (video) (opens in a new tab)
  • CS 61B Lecture 32: Sorting III (video) (opens in a new tab)
  • CS 61B Lecture 33: Sorting V (video) (opens in a new tab)
  • CS 61B 2014-04-21: Radix Sort(video) (opens in a new tab)

• Bubble Sort (video) (opens in a new tab)

• Analyzing Bubble Sort (video) (opens in a new tab)

• Insertion Sort, Merge Sort (video) (opens in a new tab)

• Insertion Sort (video) (opens in a new tab)

• Merge Sort (video) (opens in a new tab)

• Quicksort (video) (opens in a new tab)

• Selection Sort (video) (opens in a new tab)

• Merge sort code:

  • Using output array (C) (opens in a new tab)
  • Using output array (Python) (opens in a new tab)
  • In-place (C++) (opens in a new tab)

• Quick sort code:

  • Implementation (C) (opens in a new tab)
  • Implementation (C) (opens in a new tab)
  • Implementation (Python) (opens in a new tab)

• [Review] Sorting (playlist) in 18 minutes (opens in a new tab)

  • Quick sort in 4 minutes (video) (opens in a new tab)
  • Heap sort in 4 minutes (video) (opens in a new tab)
  • Merge sort in 3 minutes (video) (opens in a new tab)
  • Bubble sort in 2 minutes (video) (opens in a new tab)
  • Selection sort in 3 minutes (video) (opens in a new tab)
  • Insertion sort in 2 minutes (video) (opens in a new tab)

• Implement:

  • Mergesort: O(n log n) average and worst case
  • Quicksort O(n log n) average case
  • Selection sort and insertion sort are both O(n^2) average and worst-case
  • For heapsort, see Heap data structure above

• Not required, but I recommended them:

  • Sedgewick - Radix Sorts (6 videos) (opens in a new tab)
    • 1. Strings in Java (opens in a new tab)
    • 2. Key Indexed Counting (opens in a new tab)
    • 3. Least Significant Digit First String Radix Sort (opens in a new tab)
    • 4. Most Significant Digit First String Radix Sort (opens in a new tab)
    • 5. 3 Way Radix Quicksort (opens in a new tab)
    • 6. Suffix Arrays (opens in a new tab)
  • Radix Sort (opens in a new tab)
  • Radix Sort (video) (opens in a new tab)
  • Radix Sort, Counting Sort (linear time given constraints) (video) (opens in a new tab)
  • Randomization: Matrix Multiply, Quicksort, Freivalds' algorithm (video) (opens in a new tab)
  • Sorting in Linear Time (video) (opens in a new tab)

As a summary, here is a visual representation of 15 sorting algorithms (opens in a new tab). If you need more detail on this subject, see the "Sorting" section in Additional Detail on Some Subjects

Graphs

Graphs can be used to represent many problems in computer science, so this section is long, like trees and sorting.

• Notes:

  • There are 4 basic ways to represent a graph in memory:
    • objects and pointers
    • adjacency matrix
    • adjacency list
    • adjacency map
  • Familiarize yourself with each representation and its pros & cons
  • BFS and DFS - know their computational complexity, their trade-offs, and how to implement them in real code
  • When asked a question, look for a graph-based solution first, then move on if none

• MIT(videos):

  • Breadth-First Search (opens in a new tab)
  • Depth-First Search (opens in a new tab)

• Skiena Lectures - great intro:

  • CSE373 2020 - Lecture 10 - Graph Data Structures (video) (opens in a new tab)
  • CSE373 2020 - Lecture 11 - Graph Traversal (video) (opens in a new tab)
  • CSE373 2020 - Lecture 12 - Depth First Search (video) (opens in a new tab)
  • CSE373 2020 - Lecture 13 - Minimum Spanning Trees (video) (opens in a new tab)
  • CSE373 2020 - Lecture 14 - Minimum Spanning Trees (con't) (video) (opens in a new tab)
  • CSE373 2020 - Lecture 15 - Graph Algorithms (con't 2) (video) (opens in a new tab)

• Graphs (review and more):

  • 6.006 Single-Source Shortest Paths Problem (video) (opens in a new tab)
  • 6.006 Dijkstra (video) (opens in a new tab)
  • 6.006 Bellman-Ford (video) (opens in a new tab)
  • 6.006 Speeding Up Dijkstra (video) (opens in a new tab)
  • Aduni: Graph Algorithms I - Topological Sorting, Minimum Spanning Trees, Prim's Algorithm - Lecture 6 (video) (opens in a new tab)
  • Aduni: Graph Algorithms II - DFS, BFS, Kruskal's Algorithm, Union Find Data Structure - Lecture 7 (video) (opens in a new tab)
  • Aduni: Graph Algorithms III: Shortest Path - Lecture 8 (video) (opens in a new tab)
  • Aduni: Graph Alg. IV: Intro to geometric algorithms - Lecture 9 (video) (opens in a new tab)
  • CS 61B 2014: Weighted graphs (video) (opens in a new tab)
  • Greedy Algorithms: Minimum Spanning Tree (video) (opens in a new tab)
  • Strongly Connected Components Kosaraju's Algorithm Graph Algorithm (video) (opens in a new tab)
  • [Review] Shortest Path Algorithms (playlist) in 16 minutes (video) (opens in a new tab)
  • [Review] Minimum Spanning Trees (playlist) in 4 minutes (video) (opens in a new tab)

• Full Coursera Course:

  • Algorithms on Graphs (video) (opens in a new tab)

• I'll implement:

  • DFS with adjacency list (recursive)
  • DFS with adjacency list (iterative with stack)
  • DFS with adjacency matrix (recursive)
  • DFS with adjacency matrix (iterative with stack)
  • BFS with adjacency list
  • BFS with adjacency matrix
  • single-source shortest path (Dijkstra)
  • minimum spanning tree
  • DFS-based algorithms (see Aduni videos above):
    • check for a cycle (needed for topological sort, since we'll check for the cycle before starting)
    • topological sort
    • count connected components in a graph
    • list strongly connected components
    • check for bipartite graph

Even More Knowledge

• Recursion

  • Stanford lectures on recursion & backtracking:
    • Lecture 8 | Programming Abstractions (video) (opens in a new tab)
    • Lecture 9 | Programming Abstractions (video) (opens in a new tab)
    • Lecture 10 | Programming Abstractions (video) (opens in a new tab)
    • Lecture 11 | Programming Abstractions (video) (opens in a new tab)
  • When it is appropriate to use it?
  • How is tail recursion better than not?
    • What Is Tail Recursion Why Is It So Bad? (opens in a new tab)
    • Tail Recursion (video) (opens in a new tab)
  • 5 Simple Steps for Solving Any Recursive Problem(video) (opens in a new tab)

  Backtracking Blueprint: Java (opens in a new tab) Python (opens in a new tab)

• Dynamic Programming

  • You probably won't see any dynamic programming problems in your interview, but it's worth being able to recognize a problem as being a candidate for dynamic programming.
  • This subject can be pretty difficult, as each DP soluble problem must be defined as a recursion relation, and coming up with it can be tricky.
  • I suggest looking at many examples of DP problems until you have a solid understanding of the pattern involved.
  • Videos:
    • Skiena: CSE373 2020 - Lecture 19 - Introduction to Dynamic Programming (video) (opens in a new tab)
    • Skiena: CSE373 2020 - Lecture 20 - Edit Distance (video) (opens in a new tab)
    • Skiena: CSE373 2020 - Lecture 20 - Edit Distance (continued) (video) (opens in a new tab)
    • Skiena: CSE373 2020 - Lecture 21 - Dynamic Programming (video) (opens in a new tab)
    • Skiena: CSE373 2020 - Lecture 22 - Dynamic Programming and Review (video) (opens in a new tab)
    • Simonson: Dynamic Programming 0 (starts at 59:18) (video) (opens in a new tab)
    • Simonson: Dynamic Programming I - Lecture 11 (video) (opens in a new tab)
    • Simonson: Dynamic programming II - Lecture 12 (video) (opens in a new tab)
    • List of individual DP problems (each is short): Dynamic Programming (video) (opens in a new tab)
  • Yale Lecture notes:
    • Dynamic Programming (opens in a new tab)
  • Coursera:
    • The RNA secondary structure problem (video) (opens in a new tab)
    • A dynamic programming algorithm (video) (opens in a new tab)
    • Illustrating the DP algorithm (video) (opens in a new tab)
    • Running time of the DP algorithm (video) (opens in a new tab)
    • DP vs. recursive implementation (video) (opens in a new tab)
    • Global pairwise sequence alignment (video) (opens in a new tab)
    • Local pairwise sequence alignment (video) (opens in a new tab)

• Design patterns

  • Quick UML review (video) (opens in a new tab)
  • Learn these patterns:
    • strategy
    • singleton
    • adapter
    • prototype
    • decorator
    • visitor
    • factory, abstract factory
    • facade
    • observer
    • proxy
    • delegate
    • command
    • state
    • memento
    • iterator
    • composite
    • flyweight
  • Series of videos (27 videos) (opens in a new tab)
  • Book: Head First Design Patterns (opens in a new tab)
    • I know the canonical book is "Design Patterns: Elements of Reusable Object-Oriented Software", but Head First is great for beginners to OO.
  • Handy reference: 101 Design Patterns & Tips for Developers (opens in a new tab)

• Combinatorics (n choose k) & Probability

  • Math Skills: How to find Factorial, Permutation, and Combination (Choose) (video) (opens in a new tab)
  • Make School: Probability (video) (opens in a new tab)
  • Make School: More Probability and Markov Chains (video) (opens in a new tab)
  • Khan Academy:
    • Course layout:
      • Basic Theoretical Probability (opens in a new tab)
    • Just the videos - 41 (each are simple and each are short):
      • Probability Explained (video) (opens in a new tab)

• NP, NP-Complete and Approximation Algorithms

  • Know about the most famous classes of NP-complete problems, such as the traveling salesman and the knapsack problem, and be able to recognize them when an interviewer asks you them in disguise.
  • Know what NP-complete means.
  • Computational Complexity (video) (opens in a new tab)
  • Simonson:
    • Greedy Algs. II & Intro to NP-Completeness (video) (opens in a new tab)
    • NP Completeness II & Reductions (video) (opens in a new tab)
    • NP Completeness III (Video) (opens in a new tab)
    • NP Completeness IV (video) (opens in a new tab)
  • Skiena:
    • CSE373 2020 - Lecture 23 - NP-Completeness (video) (opens in a new tab)
    • CSE373 2020 - Lecture 24 - Satisfiability (video) (opens in a new tab)
    • CSE373 2020 - Lecture 25 - More NP-Completeness (video) (opens in a new tab)
    • CSE373 2020 - Lecture 26 - NP-Completeness Challenge (video) (opens in a new tab)
  • Complexity: P, NP, NP-completeness, Reductions (video) (opens in a new tab)
  • Complexity: Approximation Algorithms (video) (opens in a new tab)
  • Complexity: Fixed-Parameter Algorithms (video) (opens in a new tab)
  • Peter Norvig discusses near-optimal solutions to the traveling salesman problem:
    • Jupyter Notebook (opens in a new tab)
  • Pages 1048 - 1140 in CLRS if you have it.

• How computers process a program

  • How CPU executes a program (video) (opens in a new tab)
  • How computers calculate - ALU (video) (opens in a new tab)
  • Registers and RAM (video) (opens in a new tab)
  • The Central Processing Unit (CPU) (video) (opens in a new tab)
  • Instructions and Programs (video) (opens in a new tab)

• Caches

  • LRU cache:
    • The Magic of LRU Cache (100 Days of Google Dev) (video) (opens in a new tab)
    • Implementing LRU (video) (opens in a new tab)
    • LeetCode - 146 LRU Cache (C++) (video) (opens in a new tab)
  • CPU cache:
    • MIT 6.004 L15: The Memory Hierarchy (video) (opens in a new tab)
    • MIT 6.004 L16: Cache Issues (video) (opens in a new tab)

• Processes and Threads

  • Computer Science 162 - Operating Systems (25 videos):
    • for processes and threads see videos 1-11
    • Operating Systems and System Programming (video) (opens in a new tab)
  • What Is The Difference Between A Process And A Thread? (opens in a new tab)
  • Covers:
    • Processes, Threads, Concurrency issues
      • Difference between processes and threads
      • Processes
      • Threads
      • Locks
      • Mutexes
      • Semaphores
      • Monitors
      • How do they work?
      • Deadlock
      • Livelock
    • CPU activity, interrupts, context switching
    • Modern concurrency constructs with multicore processors
    • Paging, segmentation, and virtual memory (video) (opens in a new tab)
    • Interrupts (video) (opens in a new tab)
    • Process resource needs (memory: code, static storage, stack, heap, and also file descriptors, i/o)
    • Thread resource needs (shares above (minus stack) with other threads in the same process but each has its own PC, stack counter, registers, and stack)
    • Forking is really copy on write (read-only) until the new process writes to memory, then it does a full copy.
    • Context switching
      • How context switching is initiated by the operating system and underlying hardware? (opens in a new tab)
  • threads in C++ (series - 10 videos) (opens in a new tab)
  • CS 377 Spring '14: Operating Systems from University of Massachusetts (opens in a new tab)
  • concurrency in Python (videos):
    • Short series on threads (opens in a new tab)
    • Python Threads (opens in a new tab)
    • Understanding the Python GIL (2010) (opens in a new tab)
      • reference (opens in a new tab)
    • David Beazley - Python Concurrency From the Ground Up LIVE! - PyCon 2015 (opens in a new tab)
    • Keynote David Beazley - Topics of Interest (Python Asyncio) (opens in a new tab)
    • Mutex in Python (opens in a new tab)

• Testing

  • To cover:
    • how unit testing works
    • what are mock objects
    • what is integration testing
    • what is dependency injection
  • Agile Software Testing with James Bach (video) (opens in a new tab)
  • Open Lecture by James Bach on Software Testing (video) (opens in a new tab)
  • Steve Freeman - Test-Driven Development (that’s not what we meant) (video) (opens in a new tab)
    • slides (opens in a new tab)
  • Dependency injection:
    • video (opens in a new tab)
    • Tao Of Testing (opens in a new tab)
  • How to write tests (opens in a new tab)

• String searching & manipulations

  • Sedgewick - Suffix Arrays (video) (opens in a new tab)
  • Sedgewick - Substring Search (videos) (opens in a new tab)
    • 1. Introduction to Substring Search (opens in a new tab)
    • 2. Brute-Force Substring Search (opens in a new tab)
    • 3. Knuth-Morris Pratt (opens in a new tab)
    • 4. Boyer-Moore (opens in a new tab)
    • 5. Rabin-Karp (opens in a new tab)
  • Search pattern in a text (video) (opens in a new tab)

  If you need more detail on this subject, see the "String Matching" section in Additional Detail on Some Subjects.

• Tries

  • Note there are different kinds of tries. Some have prefixes, some don't, and some use strings instead of bits to track the path
  • I read through the code, but will not implement
  • Sedgewick - Tries (3 videos) (opens in a new tab)
    • 1. R Way Tries (opens in a new tab)
    • 2. Ternary Search Tries (opens in a new tab)
    • 3. Character Based Operations (opens in a new tab)
  • Notes on Data Structures and Programming Techniques (opens in a new tab)
  • Short course videos:
    • Introduction To Tries (video) (opens in a new tab)
    • Performance Of Tries (video) (opens in a new tab)
    • Implementing A Trie (video) (opens in a new tab)
  • The Trie: A Neglected Data Structure (opens in a new tab)
  • TopCoder - Using Tries (opens in a new tab)
  • Stanford Lecture (real-world use case) (video) (opens in a new tab)
  • MIT, Advanced Data Structures, Strings (can get pretty obscure about halfway through) (video) (opens in a new tab)

• Floating Point Numbers

  • simple 8-bit: Representation of Floating Point Numbers - 1 (video - there is an error in calculations - see video description) (opens in a new tab)

• Unicode

  • The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (opens in a new tab)
  • What Every Programmer Absolutely, Positively Needs To Know About Encodings And Character Sets To Work With Text (opens in a new tab)

• Endianness

  • Big And Little Endian (opens in a new tab)
  • Big Endian Vs Little Endian (video) (opens in a new tab)
  • Big And Little Endian Inside/Out (video) (opens in a new tab)
    • Very technical talk for kernel devs. Don't worry if most is over your head.
    • The first half is enough.

• Networking

  • If you have networking experience or want to be a reliability engineer or operations engineer, expect questions
  • Otherwise, this is just good to know
  • Khan Academy (opens in a new tab)
  • UDP and TCP: Comparison of Transport Protocols (video) (opens in a new tab)
  • TCP/IP and the OSI Model Explained! (video) (opens in a new tab)
  • Packet Transmission across the Internet. Networking & TCP/IP tutorial. (video) (opens in a new tab)
  • HTTP (video) (opens in a new tab)
  • SSL and HTTPS (video) (opens in a new tab)
  • SSL/TLS (video) (opens in a new tab)
  • HTTP 2.0 (video) (opens in a new tab)
  • Video Series (21 videos) (video) (opens in a new tab)
  • Subnetting Demystified - Part 5 CIDR Notation (video) (opens in a new tab)
  • Sockets:
    • Java - Sockets - Introduction (video) (opens in a new tab)
    • Socket Programming (video) (opens in a new tab)

Final Review

This section will have shorter videos that you can watch pretty quickly to review most of the important concepts. It's nice if you want a refresher often.

• Series of 2-3 minutes short subject videos (23 videos)
  • Videos (opens in a new tab)
• Series of 2-5 minutes short subject videos - Michael Sambol (48 videos):
  • Videos (opens in a new tab)
  • Code Examples (opens in a new tab)
• Sedgewick Videos - Algorithms I (opens in a new tab)
• Sedgewick Videos - Algorithms II (opens in a new tab)

Update Your Resume

• See Resume prep information in the books: "Cracking The Coding Interview" and "Programming Interviews Exposed"
• "This Is What A GOOD Resume Should Look Like" by Gayle McDowell (author of Cracking the Coding Interview) (opens in a new tab),
  • Note by the author: "This is for a US-focused resume. CVs for India and other countries have different expectations, although many of the points will be the same."
• "Step-by-step resume guide" by Tech Interview Handbook (opens in a new tab)
  • Detailed guide on how to set up your resume from scratch, write effective resume content, optimize it, and test your resume

Interview Process & General Interview Prep

• How to Pass the Engineering Interview in 2021 (opens in a new tab)
• Demystifying Tech Recruiting (opens in a new tab)
• How to Get a Job at the Big 4:
  • How to Get a Job at the Big 4 - Amazon, Facebook, Google & Microsoft (video) (opens in a new tab)
  • How to Get a Job at the Big 4.1 (Follow-up video) (opens in a new tab)
• Cracking The Coding Interview Set 1:
  • Gayle L McDowell - Cracking The Coding Interview (video) (opens in a new tab)
  • Cracking the Coding Interview with Author Gayle Laakmann McDowell (video) (opens in a new tab)
• Cracking the Facebook Coding Interview:
  • The Approach (opens in a new tab)
  • Problem Walkthrough (opens in a new tab)
• Prep Courses:
  • Python for Data Structures, Algorithms, and Interviews (paid course) (opens in a new tab):
    • A Python-centric interview prep course that covers data structures, algorithms, mock interviews, and much more.
  • Intro to Data Structures and Algorithms using Python (Udacity free course) (opens in a new tab):
    • A free Python-centric data structures and algorithms course.
  • Data Structures and Algorithms Nanodegree! (Udacity paid Nanodegree) (opens in a new tab):
    • Get hands-on practice with over 100 data structures and algorithm exercises and guidance from a dedicated mentor to help prepare you for interviews and on-the-job scenarios.
  • Grokking the Behavioral Interview (Educative free course) (opens in a new tab):
    • Many times, it’s not your technical competency that holds you back from landing your dream job, it’s how you perform on the behavioral interview.
  • AlgoMonster (paid course with free content) (opens in a new tab):
    • The crash course for LeetCode. Covers all the patterns condensed from thousands of questions.

Mock Interviews:

• Gainlo.co: Mock interviewers from big companies (opens in a new tab) - I used this and it helped me relax for the phone screen and on-site interview
• Pramp: Mock interviews from/with peers (opens in a new tab) - a peer-to-peer model to practice interviews
• interviewing.io: Practice mock interview with senior engineers (opens in a new tab) - anonymous algorithmic/systems design interviews with senior engineers from FAANG anonymously
• Meetapro: Mock interviews with top FAANG interviewers (opens in a new tab) - an Airbnb-style mock interview/coaching platform.
• Hello Interview: Mock Interviews with Expert Coaches and AI (opens in a new tab) - interview directly with AI or with FAANG staff engineers and managers.
• Codemia: Practice system design problems with AI or community solutions and feedback (opens in a new tab) - Practice system design problems via AI practice tool. Share your solution with the community to get human feedback as well.

Be thinking of for when the interview comes

Think of about 20 interview questions you'll get, along with the lines of the items below. Have at least one answer for each. Have a story, not just data, about something you accomplished.

• Why do you want this job?
• What's a tough problem you've solved?
• Biggest challenges faced?
• Best/worst designs seen?
• Ideas for improving an existing product
• How do you work best, as an individual and as part of a team?
• Which of your skills or experiences would be assets in the role and why?
• What did you most enjoy at [job x / project y]?
• What was the biggest challenge you faced at [job x / project y]?
• What was the hardest bug you faced at [job x / project y]?
• What did you learn at [job x / project y]?
• What would you have done better at [job x / project y]?

Have questions for the interviewer

Some of mine (I already may know the answers, but want their opinion or team perspective):

• How large is your team?
• What does your dev cycle look like? Do you do waterfall/sprints/agile?
• Are rushes to deadlines common? Or is there flexibility?
• How are decisions made in your team?
• How many meetings do you have per week?
• Do you feel your work environment helps you concentrate?
• What are you working on?
• What do you like about it?
• What is the work life like?
• How is the work/life balance?

Once You've Got The Job

Congratulations!

Keep learning.

You're never really done.

Everything below this point is optional. It is NOT needed for an entry-level interview. However, by studying these, you'll get greater exposure to more CS concepts and will be better prepared for any software engineering job. You'll be a much more well-rounded software engineer.

Additional Books

These are here so you can dive into a topic you find interesting.

• The Unix Programming Environment (opens in a new tab)
  • An oldie but a goodie
• The Linux Command Line: A Complete Introduction (opens in a new tab)
  • A modern option
• TCP/IP Illustrated Series (opens in a new tab)
• Head First Design Patterns (opens in a new tab)
  • A gentle introduction to design patterns
• Design Patterns: Elements of Reusable Object-Oriented Software (opens in a new tab)
  • AKA the "Gang Of Four" book or GOF
  • The canonical design patterns book
• Algorithm Design Manual (opens in a new tab) (Skiena)
  • As a review and problem-recognition
  • The algorithm catalog portion is well beyond the scope of difficulty you'll get in an interview
  • This book has 2 parts:
    • Class textbook on data structures and algorithms
      • Pros:
        • Is a good review as any algorithms textbook would be
        • Nice stories from his experiences solving problems in industry and academia
        • Code examples in C
      • Cons:
        • Can be as dense or impenetrable as CLRS, and in some cases, CLRS may be a better alternative for some subjects
        • Chapters 7, 8, and 9 can be painful to try to follow, as some items are not explained well or require more brain than I have
        • Don't get me wrong: I like Skiena, his teaching style, and mannerisms, but I may not be Stony Brook material
    • Algorithm catalog:
      • This is the real reason you buy this book.
      • This book is better as an algorithm reference, and not something you read cover to cover.
  • Can rent it on Kindle
  • Answers:
    • Solutions (opens in a new tab)
  • Errata (opens in a new tab)
• Algorithm (opens in a new tab) (Jeff Erickson)
• Write Great Code: Volume 1: Understanding the Machine (opens in a new tab)
  • The book was published in 2004, and is somewhat outdated, but it's a terrific resource for understanding a computer in brief
  • The author invented HLA (opens in a new tab), so take mentions and examples in HLA with a grain of salt. Not widely used, but decent examples of what assembly looks like
  • These chapters are worth the read to give you a nice foundation:
    • Chapter 2 - Numeric Representation
    • Chapter 3 - Binary Arithmetic and Bit Operations
    • Chapter 4 - Floating-Point Representation
    • Chapter 5 - Character Representation
    • Chapter 6 - Memory Organization and Access
    • Chapter 7 - Composite Data Types and Memory Objects
    • Chapter 9 - CPU Architecture
    • Chapter 10 - Instruction Set Architecture
    • Chapter 11 - Memory Architecture and Organization
• Introduction to Algorithms (opens in a new tab)
  • Important: Reading this book will only have limited value. This book is a great review of algorithms and data structures, but won't teach you how to write good code. You have to be able to code a decent solution efficiently
  • AKA CLR, sometimes CLRS, because Stein was late to the game
• Computer Architecture, Sixth Edition: A Quantitative Approach (opens in a new tab)
  • For a richer, more up-to-date (2017), but longer treatment

System Design, Scalability, Data Handling

You can expect system design questions if you have 4+ years of experience.

• Scalability and System Design are very large topics with many topics and resources, since there is a lot to consider when designing a software/hardware system that can scale. Expect to spend quite a bit of time on this
• Considerations:
  • Scalability
    • Distill large data sets to single values
    • Transform one data set to another
    • Handling obscenely large amounts of data
  • System design
    • features sets
    • interfaces
    • class hierarchies
    • designing a system under certain constraints
    • simplicity and robustness
    • tradeoffs
    • performance analysis and optimization
• START HERE: The System Design Primer (opens in a new tab)
• System Design from HiredInTech (opens in a new tab)
• How Do I Prepare To Answer Design Questions In A Technical Interview? (opens in a new tab)
• 8 steps guide to ace your system design interview (opens in a new tab)
• Database Normalization - 1NF, 2NF, 3NF and 4NF (video) (opens in a new tab)
• System Design Interview (opens in a new tab) - There are a lot of resources in this one. Look through the articles and examples. I put some of them below
• How to ace a systems design interview (opens in a new tab)
• Numbers Everyone Should Know (opens in a new tab)
• How long does it take to make a context switch? (opens in a new tab)
• Transactions Across Datacenters (video) (opens in a new tab)
• A plain English introduction to CAP Theorem (opens in a new tab)
• MIT 6.824: Distributed Systems, Spring 2020 (20 videos) (opens in a new tab)
• Consensus Algorithms:
  • Paxos - Paxos Agreement - Computerphile (video) (opens in a new tab)
  • Raft - An Introduction to the Raft Distributed Consensus Algorithm (video) (opens in a new tab)
    • Easy-to-read paper (opens in a new tab)
    • Infographic (opens in a new tab)
• Consistent Hashing (opens in a new tab)
• NoSQL Patterns (opens in a new tab)
• Scalability:
  • You don't need all of these. Just pick a few that interest you.
  • Great overview (video) (opens in a new tab)
  • Short series:
    • Clones (opens in a new tab)
    • Database (opens in a new tab)
    • Cache (opens in a new tab)
    • Asynchronism (opens in a new tab)
  • Scalable Web Architecture and Distributed Systems (opens in a new tab)
  • Fallacies of Distributed Computing Explained (opens in a new tab)
  • Jeff Dean - Building Software Systems At Google and Lessons Learned (video) (opens in a new tab)
  • Introduction to Architecting Systems for Scale (opens in a new tab)
  • Scaling mobile games to a global audience using App Engine and Cloud Datastore (video) (opens in a new tab)
  • How Google Does Planet-Scale Engineering for Planet-Scale Infra (video) (opens in a new tab)
  • The Importance of Algorithms (opens in a new tab)
  • Sharding (opens in a new tab)
  • Engineering for the Long Game - Astrid Atkinson Keynote(video) (opens in a new tab)
  • 7 Years Of YouTube Scalability Lessons In 30 Minutes (opens in a new tab)
    • video (opens in a new tab)
  • How PayPal Scaled To Billions Of Transactions Daily Using Just 8VMs (opens in a new tab)
  • How to Remove Duplicates in Large Datasets (opens in a new tab)
  • A look inside Etsy's scale and engineering culture with Jon Cowie (video) (opens in a new tab)
  • What Led Amazon to its Own Microservices Architecture (opens in a new tab)
  • To Compress Or Not To Compress, That Was Uber's Question (opens in a new tab)
  • When Should Approximate Query Processing Be Used? (opens in a new tab)
  • Google's Transition From Single Datacenter To Failover, To A Native Multihomed Architecture (opens in a new tab)
  • The Image Optimization Technology That Serves Millions Of Requests Per Day (opens in a new tab)
  • A Patreon Architecture Short (opens in a new tab)
  • Tinder: How Does One Of The Largest Recommendation Engines Decide Who You'll See Next? (opens in a new tab)
  • Design Of A Modern Cache (opens in a new tab)
  • Live Video Streaming At Facebook Scale (opens in a new tab)
  • A Beginner's Guide To Scaling To 11 Million+ Users On Amazon's AWS (opens in a new tab)
  • A 360 Degree View Of The Entire Netflix Stack (opens in a new tab)
  • Latency Is Everywhere And It Costs You Sales - How To Crush It (opens in a new tab)
  • What Powers Instagram: Hundreds of Instances, Dozens of Technologies (opens in a new tab)
  • Salesforce Architecture - How They Handle 1.3 Billion Transactions A Day (opens in a new tab)
  • ESPN's Architecture At Scale - Operating At 100,000 Duh Nuh Nuhs Per Second (opens in a new tab)
  • See "Messaging, Serialization, and Queueing Systems" way below for info on some of the technologies that can glue services together
  • Twitter:
    • O'Reilly MySQL CE 2011: Jeremy Cole, "Big and Small Data at @Twitter" (video) (opens in a new tab)
    • Timelines at Scale (opens in a new tab)
  • For even more, see the "Mining Massive Datasets" video series in the Video Series section
• Practicing the system design process: Here are some ideas to try working through on paper, each with some documentation on how it was handled in the real world:
  • review: The System Design Primer (opens in a new tab)
  • System Design from HiredInTech (opens in a new tab)
  • cheat sheet (opens in a new tab)
  • flow:
    1. Understand the problem and scope:
       • Define the use cases, with the interviewer's help
       • Suggest additional features
       • Remove items that the interviewer deems out of scope
       • Assume high availability is required, add as a use case
    2. Think about constraints:
       • Ask how many requests per month
       • Ask how many requests per second (they may volunteer it or make you do the math)
       • Estimate reads vs. writes percentage
       • Keep the 80/20 rule in mind when estimating
       • How much data is written per second
       • Total storage required over 5 years
       • How much data read per second
    3. Abstract design:
       • Layers (service, data, caching)
       • Infrastructure: load balancing, messaging
       • Rough overview of any key algorithm that drives the service
       • Consider bottlenecks and determine solutions
  • Exercises:
    • Design a random unique ID generation system (opens in a new tab)
    • Design a key-value database (opens in a new tab)
    • Design a picture sharing system (opens in a new tab)
    • Design a recommendation system (opens in a new tab)
    • Design a URL-shortener system: copied from above (opens in a new tab)
    • Design a cache system (opens in a new tab)

Additional Learning

I added them to help you become a well-rounded software engineer and to be aware of certain technologies and algorithms, so you'll have a bigger toolbox.

• Compilers

  • How a Compiler Works in ~1 minute (video) (opens in a new tab)
  • Harvard CS50 - Compilers (video) (opens in a new tab)
  • C++ (video) (opens in a new tab)
  • Understanding Compiler Optimization (C++) (video) (opens in a new tab)

• Emacs and vi(m)

  • Familiarize yourself with a UNIX-based code editor
  • vi(m):
    • Editing With Vim 01 - Installation, Setup, and The Modes (video) (opens in a new tab)
    • VIM Adventures (opens in a new tab)
    • set of 4 videos:
      • The vi/vim editor - Lesson 1 (opens in a new tab)
      • The vi/vim editor - Lesson 2 (opens in a new tab)
      • The vi/vim editor - Lesson 3 (opens in a new tab)
      • The vi/vim editor - Lesson 4 (opens in a new tab)
    • Using Vi Instead of Emacs (opens in a new tab)
  • emacs:
    • Basics Emacs Tutorial (video) (opens in a new tab)
    • set of 3 (videos):
      • Emacs Tutorial (Beginners) -Part 1- File commands, cut/copy/paste, cursor commands (opens in a new tab)
      • Emacs Tutorial (Beginners) -Part 2- Buffer management, search, M-x grep and rgrep modes (opens in a new tab)
      • Emacs Tutorial (Beginners) -Part 3- Expressions, Statements, ~/.emacs file, and packages (opens in a new tab)
    • Evil Mode: Or, How I Learned to Stop Worrying and Love Emacs (video) (opens in a new tab)
    • Writing C Programs With Emacs (opens in a new tab)
  • The Absolute Beginner's Guide to Emacs (video by David Wilson) (opens in a new tab)
  • The Absolute Beginner's Guide to Emacs (notes by David Wilson) (opens in a new tab)

• Unix command line tools

  • I filled in the list below from good tools.
  • bash
  • cat
  • grep
  • sed
  • awk
  • curl or wget
  • sort
  • tr
  • uniq
  • strace (opens in a new tab)
  • tcpdump (opens in a new tab)

• Information theory (videos)

  • Khan Academy (opens in a new tab)
  • More about Markov processes:
    • Core Markov Text Generation (opens in a new tab)
    • Core Implementing Markov Text Generation (opens in a new tab)
    • Project = Markov Text Generation Walk Through (opens in a new tab)
  • See more in the MIT 6.050J Information and Entropy series below

• Parity & Hamming Code (videos)

  • Intro (opens in a new tab)
  • Parity (opens in a new tab)
  • Hamming Code:
    • Error detection (opens in a new tab)
    • Error correction (opens in a new tab)
  • Error Checking (opens in a new tab)

• Entropy

  • Also see the videos below
  • Make sure to watch information theory videos first
  • Information Theory, Claude Shannon, Entropy, Redundancy, Data Compression & Bits (video) (opens in a new tab)

• Cryptography

  • Also see the videos below
  • Make sure to watch information theory videos first
  • Khan Academy Series (opens in a new tab)
  • Cryptography: Hash Functions (opens in a new tab)
  • Cryptography: Encryption (opens in a new tab)

• Compression

  • Make sure to watch information theory videos first
  • Computerphile (videos):
    • Compression (opens in a new tab)
    • Entropy in Compression (opens in a new tab)
    • Upside Down Trees (Huffman Trees) (opens in a new tab)
    • EXTRA BITS/TRITS - Huffman Trees (opens in a new tab)
    • Elegant Compression in Text (The LZ 77 Method) (opens in a new tab)
    • Text Compression Meets Probabilities (opens in a new tab)
  • Compressor Head videos (opens in a new tab)
  • (optional) Google Developers Live: GZIP is not enough! (opens in a new tab)

• Computer Security

  • MIT (23 videos) (opens in a new tab)
    • Introduction, Threat Models (opens in a new tab)
    • Control Hijacking Attacks (opens in a new tab)
    • Buffer Overflow Exploits and Defenses (opens in a new tab)
    • Privilege Separation (opens in a new tab)
    • Capabilities (opens in a new tab)
    • Sandboxing Native Code (opens in a new tab)
    • Web Security Model (opens in a new tab)
    • Securing Web Applications (opens in a new tab)
    • Symbolic Execution (opens in a new tab)
    • Network Security (opens in a new tab)
    • Network Protocols (opens in a new tab)
    • Side-Channel Attacks (opens in a new tab)

• Garbage collection

  • GC in Python (video) (opens in a new tab)
  • Deep Dive Java: Garbage Collection is Good! (opens in a new tab)
  • Deep Dive Python: Garbage Collection in CPython (video) (opens in a new tab)

• Parallel Programming

  • Coursera (Scala) (opens in a new tab)
  • Efficient Python for High-Performance Parallel Computing (video) (opens in a new tab)

• Messaging, Serialization, and Queueing Systems

  • Thrift (opens in a new tab)
    • Tutorial (opens in a new tab)
  • Protocol Buffers (opens in a new tab)
    • Tutorials (opens in a new tab)
  • gRPC (opens in a new tab)
    • gRPC 101 for Java Developers (video) (opens in a new tab)
  • Redis (opens in a new tab)
    • Tutorial (opens in a new tab)
  • Amazon SQS (queue) (opens in a new tab)
  • Amazon SNS (pub-sub) (opens in a new tab)
  • RabbitMQ (opens in a new tab)
    • Get Started (opens in a new tab)
  • Celery (opens in a new tab)
    • First Steps With Celery (opens in a new tab)
  • ZeroMQ (opens in a new tab)
    • Intro - Read The Manual (opens in a new tab)
  • ActiveMQ (opens in a new tab)
  • Kafka (opens in a new tab)
  • MessagePack (opens in a new tab)
  • Avro (opens in a new tab)

• A*

  • A Search Algorithm (opens in a new tab)
  • A* Pathfinding (E01: algorithm explanation) (video) (opens in a new tab)

• Fast Fourier Transform

  • An Interactive Guide To The Fourier Transform (opens in a new tab)
  • What is a Fourier transform? What is it used for? (opens in a new tab)
  • What is the Fourier Transform? (video) (opens in a new tab)
  • Divide & Conquer: FFT (video) (opens in a new tab)
  • Understanding The FFT (opens in a new tab)

• Bloom Filter

  • Given a Bloom filter with m bits and k hashing functions, both insertion and membership testing are O(k)
  • Bloom Filters (video) (opens in a new tab)
  • Bloom Filters | Mining of Massive Datasets | Stanford University (video) (opens in a new tab)
  • Tutorial (opens in a new tab)
  • How To Write A Bloom Filter App (opens in a new tab)

• HyperLogLog

  • How To Count A Billion Distinct Objects Using Only 1.5KB Of Memory (opens in a new tab)

• Locality-Sensitive Hashing

  • Used to determine the similarity of documents
  • The opposite of MD5 or SHA which are used to determine if 2 documents/strings are exactly the same
  • Simhashing (hopefully) made simple (opens in a new tab)

• van Emde Boas Trees

  • Divide & Conquer: van Emde Boas Trees (video) (opens in a new tab)
  • MIT Lecture Notes (opens in a new tab)

• Augmented Data Structures

  • CS 61B Lecture 39: Augmenting Data Structures (opens in a new tab)

• Balanced search trees

  • Know at least one type of balanced binary tree (and know how it's implemented):

  • "Among balanced search trees, AVL and 2/3 trees are now passé and red-black trees seem to be more popular. A particularly interesting self-organizing data structure is the splay tree, which uses rotations to move any accessed key to the root." - Skiena

  • Of these, I chose to implement a splay tree. From what I've read, you won't implement a balanced search tree in your interview. But I wanted exposure to coding one up and let's face it, splay trees are the bee's knees. I did read a lot of red-black tree code

    • Splay tree: insert, search, delete functions If you end up implementing a red/black tree try just these:
    • Search and insertion functions, skipping delete

  • I want to learn more about B-Tree since it's used so widely with very large data sets

  • Self-balancing binary search tree (opens in a new tab)

  • AVL trees

    • In practice: From what I can tell, these aren't used much in practice, but I could see where they would be: The AVL tree is another structure supporting O(log n) search, insertion, and removal. It is more rigidly balanced than red–black trees, leading to slower insertion and removal but faster retrieval. This makes it attractive for data structures that may be built once and loaded without reconstruction, such as language dictionaries (or program dictionaries, such as the opcodes of an assembler or interpreter)
    • MIT AVL Trees / AVL Sort (video) (opens in a new tab)
    • AVL Trees (video) (opens in a new tab)
    • AVL Tree Implementation (video) (opens in a new tab)
    • Split And Merge (opens in a new tab)
    • [Review] AVL Trees (playlist) in 19 minutes (video) (opens in a new tab)

  • Splay trees

    • In practice: Splay trees are typically used in the implementation of caches, memory allocators, routers, garbage collectors, data compression, ropes (replacement of string used for long text strings), in Windows NT (in the virtual memory, networking and file system code) etc
    • CS 61B: Splay Trees (video) (opens in a new tab)
    • MIT Lecture: Splay Trees:
      • Gets very mathy, but watch the last 10 minutes for sure.
      • Video (opens in a new tab)

  • Red/black trees

    • These are a translation of a 2-3 tree (see below).
    • In practice: Red–black trees offer worst-case guarantees for insertion time, deletion time, and search time. Not only does this make them valuable in time-sensitive applications such as real-time applications, but it makes them valuable building blocks in other data structures that provide worst-case guarantees; for example, many data structures used in computational geometry can be based on red-black trees, and the Completely Fair Scheduler used in current Linux kernels uses red–black trees. In version 8 of Java, the Collection HashMap has been modified such that instead of using a LinkedList to store identical elements with poor hashcodes, a Red-Black tree is used
    • Aduni - Algorithms - Lecture 4 (link jumps to the starting point) (video) (opens in a new tab)
    • Aduni - Algorithms - Lecture 5 (video) (opens in a new tab)
    • Red-Black Tree (opens in a new tab)
    • An Introduction To Binary Search And Red Black Tree (opens in a new tab)
    • [Review] Red-Black Trees (playlist) in 30 minutes (video) (opens in a new tab)

  • 2-3 search trees

    • In practice: 2-3 trees have faster inserts at the expense of slower searches (since height is more compared to AVL trees).
    • You would use 2-3 trees very rarely because its implementation involves different types of nodes. Instead, people use Red-Black trees.
    • 23-Tree Intuition and Definition (video) (opens in a new tab)
    • Binary View of 23-Tree (opens in a new tab)
    • 2-3 Trees (student recitation) (video) (opens in a new tab)

  • 2-3-4 Trees (aka 2-4 trees)

    • In practice: For every 2-4 trees, there are corresponding red–black trees with data elements in the same order. The insertion and deletion operations on 2-4 trees are also equivalent to color-flipping and rotations in red–black trees. This makes 2-4 trees an important tool for understanding the logic behind red-black trees, and this is why many introductory algorithm texts introduce 2-4 trees just before red–black trees, even though 2-4 trees are not often used in practice.
    • CS 61B Lecture 26: Balanced Search Trees (video) (opens in a new tab)
    • Bottom Up 234-Trees (video) (opens in a new tab)
    • Top Down 234-Trees (video) (opens in a new tab)

  • N-ary (K-ary, M-ary) trees

    • note: the N or K is the branching factor (max branches)
    • binary trees are a 2-ary tree, with branching factor = 2
    • 2-3 trees are 3-ary
    • K-Ary Tree (opens in a new tab)

  • B-Trees

    • Fun fact: it's a mystery, but the B could stand for Boeing, Balanced, or Bayer (co-inventor).
    • In Practice: B-trees are widely used in databases. Most modern filesystems use B-trees (or Variants). In addition to its use in databases, the B-tree is also used in filesystems to allow quick random access to an arbitrary block in a particular file. The basic problem is turning the file block address into a disk block (or perhaps to a cylinder head sector) address
    • B-Tree (opens in a new tab)
    • B-Tree Datastructure (opens in a new tab)
    • Introduction to B-Trees (video) (opens in a new tab)
    • B-Tree Definition and Insertion (video) (opens in a new tab)
    • B-Tree Deletion (video) (opens in a new tab)
    • MIT 6.851 - Memory Hierarchy Models (video) (opens in a new tab)
      • covers cache-oblivious B-Trees, very interesting data structures
      • the first 37 minutes are very technical, and may be skipped (B is block size, cache line size)
    • [Review] B-Trees (playlist) in 26 minutes (video) (opens in a new tab)

• k-D Trees

  • Great for finding a number of points in a rectangle or higher-dimensional object
  • A good fit for k-nearest neighbors
  • kNN K-d tree algorithm (video) (opens in a new tab)

• Skip lists

  • "These are somewhat of a cult data structure" - Skiena
  • Randomization: Skip Lists (video) (opens in a new tab)
  • For animations and a little more detail (opens in a new tab)

• Network Flows

  • Ford-Fulkerson in 5 minutes — Step by step example (video) (opens in a new tab)
  • Ford-Fulkerson Algorithm (video) (opens in a new tab)
  • Network Flows (video) (opens in a new tab)

• Disjoint Sets & Union Find

  • UCB 61B - Disjoint Sets; Sorting & selection (video) (opens in a new tab)
  • Sedgewick Algorithms - Union-Find (6 videos) (opens in a new tab)

• Math for Fast Processing

  • Integer Arithmetic, Karatsuba Multiplication (video) (opens in a new tab)
  • The Chinese Remainder Theorem (used in cryptography) (video) (opens in a new tab)

• Treap

  • Combination of a binary search tree and a heap
  • Treap (opens in a new tab)
  • Data Structures: Treaps explained (video) (opens in a new tab)
  • Applications in set operations (opens in a new tab)

• Linear Programming (videos)

  • Linear Programming (opens in a new tab)
  • Finding minimum cost (opens in a new tab)
  • Finding maximum value (opens in a new tab)
  • Solve Linear Equations with Python - Simplex Algorithm (opens in a new tab)

• Geometry, Convex hull (videos)

  • Graph Alg. IV: Intro to geometric algorithms - Lecture 9 (opens in a new tab)
  • Geometric Algorithms: Graham & Jarvis - Lecture 10 (opens in a new tab)
  • Divide & Conquer: Convex Hull, Median Finding (opens in a new tab)

• Discrete math

  • Computer Science 70, 001 - Spring 2015 - Discrete Mathematics and Probability Theory (opens in a new tab)
  • Discrete Mathematics by Shai Simonson (19 videos) (opens in a new tab)
  • Discrete Mathematics By IIT Ropar NPTEL (opens in a new tab)

Additional Detail on Some Subjects

I added these to reinforce some ideas already presented above, but didn't want to include them above because it's just too much. It's easy to overdo it on a subject. You want to get hired in this century, right?

• SOLID

  • Bob Martin SOLID Principles of Object Oriented and Agile Design (video) (opens in a new tab)
  • S - Single Responsibility Principle (opens in a new tab) | Single responsibility to each Object (opens in a new tab)
    • more flavor (opens in a new tab)
  • O - Open/Closed Principle (opens in a new tab) | On production level Objects are ready for extension but not for modification (opens in a new tab)
    • more flavor (opens in a new tab)
  • L - Liskov Substitution Principle (opens in a new tab) | Base Class and Derived class follow ‘IS A’ Principle (opens in a new tab)
    • more flavor (opens in a new tab)
  • I - Interface segregation principle (opens in a new tab) | Clients should not be forced to implement interfaces they don't use
    • Interface Segregation Principle in 5 minutes (video) (opens in a new tab)
    • more flavor (opens in a new tab)
  • D -Dependency Inversion principle (opens in a new tab) | Reduce the dependency In composition of objects.
    • Why Is The Dependency Inversion Principle And Why Is It Important (opens in a new tab)
    • more flavor (opens in a new tab)

• Union-Find

  • Overview (opens in a new tab)
  • Naive Implementation (opens in a new tab)
  • Trees (opens in a new tab)
  • Union By Rank (opens in a new tab)
  • Path Compression (opens in a new tab)
  • Analysis Options (opens in a new tab)

• More Dynamic Programming (videos)

  • 6.006: Dynamic Programming I: Fibonacci, Shortest Paths (opens in a new tab)
  • 6.006: Dynamic Programming II: Text Justification, Blackjack (opens in a new tab)
  • 6.006: DP III: Parenthesization, Edit Distance, Knapsack (opens in a new tab)
  • 6.006: DP IV: Guitar Fingering, Tetris, Super Mario Bros. (opens in a new tab)
  • 6.046: Dynamic Programming & Advanced DP (opens in a new tab)
  • 6.046: Dynamic Programming: All-Pairs Shortest Paths (opens in a new tab)
  • 6.046: Dynamic Programming (student recitation) (opens in a new tab)

• Advanced Graph Processing (videos)

  • Synchronous Distributed Algorithms: Symmetry-Breaking. Shortest-Paths Spanning Trees (opens in a new tab)
  • Asynchronous Distributed Algorithms: Shortest-Paths Spanning Trees (opens in a new tab)

• MIT Probability (mathy, and go slowly, which is good for mathy things) (videos):

  • MIT 6.042J - Probability Introduction (opens in a new tab)
  • MIT 6.042J - Conditional Probability (opens in a new tab)
  • MIT 6.042J - Independence (opens in a new tab)
  • MIT 6.042J - Random Variables (opens in a new tab)
  • MIT 6.042J - Expectation I (opens in a new tab)
  • MIT 6.042J - Expectation II (opens in a new tab)
  • MIT 6.042J - Large Deviations (opens in a new tab)
  • MIT 6.042J - Random Walks (opens in a new tab)

• Simonson: Approximation Algorithms (video) (opens in a new tab)

• String Matching

  • Rabin-Karp (videos):
    • Rabin Karps Algorithm (opens in a new tab)
    • Precomputing (opens in a new tab)
    • Optimization: Implementation and Analysis (opens in a new tab)
    • Table Doubling, Karp-Rabin (opens in a new tab)
    • Rolling Hashes, Amortized Analysis (opens in a new tab)
  • Knuth-Morris-Pratt (KMP):
    • TThe Knuth-Morris-Pratt (KMP) String Matching Algorithm (opens in a new tab)
  • Boyer–Moore string search algorithm
    • Boyer-Moore String Search Algorithm (opens in a new tab)
    • Advanced String Searching Boyer-Moore-Horspool Algorithms (video) (opens in a new tab)
  • Coursera: Algorithms on Strings (opens in a new tab)
    • starts off great, but by the time it gets past KMP it gets more complicated than it needs to be
    • nice explanation of tries
    • can be skipped

• Sorting

  • Stanford lectures on sorting:
    • Lecture 15 | Programming Abstractions (video) (opens in a new tab)
    • Lecture 16 | Programming Abstractions (video) (opens in a new tab)
  • Shai Simonson:
    • Algorithms - Sorting - Lecture 2 (video) (opens in a new tab)
    • Algorithms - Sorting II - Lecture 3 (video) (opens in a new tab)
  • Steven Skiena lectures on sorting:
    • CSE373 2020 - Mergesort/Quicksort (video) (opens in a new tab)
    • CSE373 2020 - Linear Sorting (video) (opens in a new tab)

• NAND To Tetris: Build a Modern Computer from First Principles (opens in a new tab)

Video Series

Sit back and enjoy.

• List of individual Dynamic Programming problems (each is short) (opens in a new tab)

• x86 Architecture, Assembly, Applications (11 videos) (opens in a new tab)

• MIT 18.06 Linear Algebra, Spring 2005 (35 videos) (opens in a new tab)

• Excellent - MIT Calculus Revisited: Single Variable Calculus (opens in a new tab)

• Skiena lectures from Algorithm Design Manual - CSE373 2020 - Analysis of Algorithms (26 videos) (opens in a new tab)

• UC Berkeley 61B (Spring 2014): Data Structures (25 videos) (opens in a new tab)

• UC Berkeley 61B (Fall 2006): Data Structures (39 videos) (opens in a new tab)

• UC Berkeley 61C: Machine Structures (26 videos) (opens in a new tab)

• OOSE: Software Dev Using UML and Java (21 videos) (opens in a new tab)

• MIT 6.004: Computation Structures (49 videos) (opens in a new tab)

• Carnegie Mellon - Computer Architecture Lectures (39 videos) (opens in a new tab)

• MIT 6.006: Intro to Algorithms (47 videos) (opens in a new tab)

• MIT 6.033: Computer System Engineering (22 videos) (opens in a new tab)

• MIT 6.034 Artificial Intelligence, Fall 2010 (30 videos) (opens in a new tab)

• MIT 6.042J: Mathematics for Computer Science, Fall 2010 (25 videos) (opens in a new tab)

• MIT 6.046: Design and Analysis of Algorithms (34 videos) (opens in a new tab)

• MIT 6.824: Distributed Systems, Spring 2020 (20 videos) (opens in a new tab)

• MIT 6.851: Advanced Data Structures (22 videos) (opens in a new tab)

• MIT 6.854: Advanced Algorithms, Spring 2016 (24 videos) (opens in a new tab)

• Harvard COMPSCI 224: Advanced Algorithms (25 videos) (opens in a new tab)

• MIT 6.858 Computer Systems Security, Fall 2014 (opens in a new tab)

• Stanford: Programming Paradigms (27 videos) (opens in a new tab)

• Introduction to Cryptography by Christof Paar (opens in a new tab)

  • Course Website along with Slides and Problem Sets (opens in a new tab)

• Mining Massive Datasets - Stanford University (94 videos) (opens in a new tab)

• Graph Theory by Sarada Herke (67 videos) (opens in a new tab)

Computer Science Courses

• Directory of Online CS Courses (opens in a new tab)
• Directory of CS Courses (many with online lectures) (opens in a new tab)

Algorithms implementation

• Multiple Algorithms implementation by Princeton University (opens in a new tab)

Papers

• Love classic papers? (opens in a new tab)
• 1978: Communicating Sequential Processes (opens in a new tab)
  • implemented in Go (opens in a new tab)
• 2003: The Google File System (opens in a new tab)
  • replaced by Colossus in 2012
• 2004: MapReduce: Simplified Data Processing on Large Clusters (opens in a new tab)
  • mostly replaced by Cloud Dataflow?
• 2006: Bigtable: A Distributed Storage System for Structured Data (opens in a new tab)
• 2006: The Chubby Lock Service for Loosely-Coupled Distributed Systems (opens in a new tab)
• 2007: Dynamo: Amazon’s Highly Available Key-value Store (opens in a new tab)
  • The Dynamo paper kicked off the NoSQL revolution
• 2007: What Every Programmer Should Know About Memory (very long, and the author encourages skipping of some sections) (opens in a new tab)
• 2012: AddressSanitizer: A Fast Address Sanity Checker:
  • paper (opens in a new tab)
  • video (opens in a new tab)
• 2013: Spanner: Google’s Globally-Distributed Database:
  • paper (opens in a new tab)
  • video (opens in a new tab)
• 2015: Continuous Pipelines at Google (opens in a new tab)
• 2015: High-Availability at Massive Scale: Building Google’s Data Infrastructure for Ads (opens in a new tab)
• 2015: How Developers Search for Code: A Case Study (opens in a new tab)
• More papers: 1,000 papers (opens in a new tab)

LICENSE

CC-BY-SA-4.0

Reference