# Java Variables

Variables in Java are containers for storing data values. Understanding variables is fundamental to Java programming.

## Types of Variables

### Primitive Variables
Java provides eight primitive data types for basic values:

- **byte**: 8-bit signed integer (-128 to 127)
- **short**: 16-bit signed integer (-32,768 to 32,767)
- **int**: 32-bit signed integer (-2³¹ to 2³¹-1)
- **long**: 64-bit signed integer (-2⁶³ to 2⁶³-1)
- **float**: 32-bit floating point
- **double**: 64-bit floating point
- **boolean**: true or false
- **char**: 16-bit Unicode character

### Reference Variables
Reference variables store addresses that point to objects in memory:

- **Objects**: Instances of classes
- **Arrays**: Collections of similar type elements
- **Strings**: Special objects for text manipulation

## Variable Declaration and Initialization

```java
// Declaration
int age;
String name;

// Initialization
age = 25;
name = "Java Developer";

// Declaration and initialization
double salary = 75000.50;
boolean isActive = true;
char grade = 'A';
```

## Variable Scope

Variables have different scopes based on where they're declared:

### Instance Variables
```java
public class Employee {
    private String name;  // Instance variable
    private int age;
    
    public Employee(String name, int age) {
        this.name = name;  // 'this' distinguishes from parameter
        this.age = age;
    }
}
```

### Static Variables
```java
public class Counter {
    private static int count = 0;  // Shared among all instances
    
    public Counter() {
        count++;
    }
    
    public static int getCount() {
        return count;
    }
}
```

### Local Variables
```java
public void calculate() {
    int result = 0;  // Local to this method
    for (int i = 0; i < 10; i++) {  // Loop variable
        result += i;
    }
    // 'i' is not accessible here
}
```

## Memory Allocation

### Stack Memory
- Stores primitive variables
- Stores references to objects
- Fast access
- Limited size
- Automatically managed

### Heap Memory
- Stores objects
- Dynamic allocation
- Larger size
- Managed by garbage collector

```java
public class MemoryExample {
    public static void main(String[] args) {
        int x = 10;           // Stack
        String name = "Java"; // Reference on stack, object on heap
        int[] numbers = new int[100]; // Reference on stack, array on heap
    }
}
```

## Best Practices

1. **Use descriptive names**: `customerAge` instead of `ca`
2. **Initialize variables**: Avoid uninitialized variables
3. **Use appropriate types**: Choose the smallest type that fits your needs
4. **Follow naming conventions**: camelCase for variables
5. **Limit scope**: Declare variables in the smallest possible scope

## Common Pitfalls

- **Uninitialized local variables**: Compiler error
- **Variable shadowing**: Hiding outer variables
- **Type mismatch**: Incompatible assignments
- **Scope issues**: Accessing variables outside their scope

Understanding variables and memory management is crucial for writing efficient and bug-free Java code.

Understanding variables in Java including primitive types, reference types, memory allocation, and scoping

Java Variables

# Java Multithreading

Multithreading enables concurrent execution of multiple threads in Java, allowing programs to perform multiple operations simultaneously.

## What is a Thread?

A thread is the smallest unit of execution within a process. Java programs can have multiple threads running concurrently.

## Creating Threads

### Method 1: Extending Thread Class
```java
class MyThread extends Thread {
    public void run() {
        System.out.println("Thread running: " + Thread.currentThread().getName());
    }
}

// Usage
MyThread thread = new MyThread();
thread.start();
```

### Method 2: Implementing Runnable Interface (Preferred)
```java
class MyTask implements Runnable {
    public void run() {
        System.out.println("Task executing: " + Thread.currentThread().getName());
    }
}

// Usage
Thread thread = new Thread(new MyTask());
thread.start();
```

### Method 3: Using Lambda Expressions
```java
Thread thread = new Thread(() -> {
    System.out.println("Lambda thread: " + Thread.currentThread().getName());
});
thread.start();
```

## Thread Lifecycle

1. **NEW**: Thread created but not started
2. **RUNNABLE**: Ready to run or running
3. **BLOCKED**: Waiting for monitor lock
4. **WAITING**: Waiting indefinitely
5. **TIMED_WAITING**: Waiting for specified time
6. **TERMINATED**: Thread execution completed

```java
Thread thread = new Thread(() -> {
    try {
        Thread.sleep(1000);  // TIMED_WAITING
    } catch (InterruptedException e) {
        e.printStackTrace();
    }
});

System.out.println(thread.getState());  // NEW
thread.start();
System.out.println(thread.getState());  // RUNNABLE
```

## Synchronization

### The Problem: Race Conditions
```java
class Counter {
    private int count = 0;
    
    public void increment() {
        count++;  // Not thread-safe!
    }
    
    public int getCount() {
        return count;
    }
}
```

### Solution: Synchronized Methods
```java
class ThreadSafeCounter {
    private int count = 0;
    
    public synchronized void increment() {
        count++;  // Thread-safe
    }
    
    public synchronized int getCount() {
        return count;
    }
}
```

### Synchronized Blocks
```java
class BankAccount {
    private double balance;
    
    public void withdraw(double amount) {
        synchronized(this) {  // Lock on this object
            if (balance >= amount) {
                balance -= amount;
            }
        }
    }
}
```

## Thread Communication

### wait() and notify()
```java
class ProducerConsumer {
    private Queue<Integer> queue = new LinkedList<>();
    private int capacity = 5;
    
    public void produce(int item) throws InterruptedException {
        synchronized(this) {
            while (queue.size() == capacity) {
                wait();  // Wait for consumer
            }
            queue.add(item);
            notify();  // Notify consumer
        }
    }
    
    public int consume() throws InterruptedException {
        synchronized(this) {
            while (queue.isEmpty()) {
                wait();  // Wait for producer
            }
            int item = queue.remove();
            notify();  // Notify producer
            return item;
        }
    }
}
```

## Concurrency Utilities

### ExecutorService
```java
ExecutorService executor = Executors.newFixedThreadPool(5);

executor.submit(() -> {
    System.out.println("Task 1");
});

executor.submit(() -> {
    System.out.println("Task 2");
});

executor.shutdown();  // Important: shutdown the executor
```

### Future and Callable
```java
ExecutorService executor = Executors.newSingleThreadExecutor();

Future<Integer> future = executor.submit(() -> {
    Thread.sleep(1000);
    return 42;
});

// Do other work while task runs
System.out.println("Doing other work...");

// Get result (blocks if not ready)
Integer result = future.get();
System.out.println("Result: " + result);

executor.shutdown();
```

## Thread Pools

### Fixed Thread Pool
```java
ExecutorService executor = Executors.newFixedThreadPool(4);
// Always 4 threads available
```

### Cached Thread Pool
```java
ExecutorService executor = Executors.newCachedThreadPool();
// Creates threads as needed, reuses idle threads
```

### Single Thread Executor
```java
ExecutorService executor = Executors.newSingleThreadExecutor();
// Only one thread at a time
```

## Common Issues

### Deadlocks
```java
// Avoid this pattern!
public void method1() {
    synchronized(lock1) {
        synchronized(lock2) {
            // Do work
        }
    }
}

public void method2() {
    synchronized(lock2) {
        synchronized(lock1) {
            // Do work
        }
    }
}
```

### Starvation
Low-priority threads may never get CPU time.

### Livelock
Threads keep responding to each other but make no progress.

## Best Practices

1. **Use ExecutorService** instead of creating threads directly
2. **Prefer immutable objects** when possible
3. **Use concurrent collections** (ConcurrentHashMap, CopyOnWriteArrayList)
4. **Minimize synchronized scope**
5. **Always shutdown ExecutorService**
6. **Handle InterruptedException properly**
7. **Use volatile for variables accessed by multiple threads**

## Modern Java Concurrency

Java 8+ introduced many improvements:
- CompletableFuture for asynchronous programming
- Parallel streams for data processing
- Improved atomic classes
- Lock interface for flexible locking

Multithreading is essential for building responsive and scalable Java applications, especially in modern multi-core systems.

Master concurrent programming with threads, synchronization, and modern concurrency utilities

Java Multithreading

# Java Collections Framework

The Java Collections Framework provides a unified architecture for representing and manipulating collections, enabling them to be manipulated independently of implementation details.

## Collection Hierarchy

```
Collection Interface
├── List Interface (Ordered, allows duplicates)
├── Set Interface (Unique elements)
└── Queue Interface (FIFO processing)

Map Interface (Key-value pairs) - Separate hierarchy
```

## List Interface

### ArrayList
```java
List<String> names = new ArrayList<>();
names.add("Alice");
names.add("Bob");
names.add("Charlie");

// Access by index
String first = names.get(0);  // "Alice"

// Remove by index
names.remove(1);  // Removes "Bob"

// Size
int size = names.size();  // 2
```

**Characteristics:**
- Dynamic array implementation
- Fast random access (O(1))
- Slow insertion/deletion in middle (O(n))
- Allows null elements

### LinkedList
```java
List<Integer> numbers = new LinkedList<>();
numbers.add(10);
numbers.add(20);
numbers.addFirst(5);   // Add at beginning
numbers.addLast(30);   // Add at end

// Access first and last elements
int first = numbers.getFirst();  // 5
int last = numbers.getLast();    // 30
```

**Characteristics:**
- Doubly-linked list implementation
- Fast insertion/deletion (O(1))
- Slow random access (O(n))
- Implements both List and Deque interfaces

## Set Interface

### HashSet
```java
Set<String> uniqueNames = new HashSet<>();
uniqueNames.add("Alice");
uniqueNames.add("Bob");
uniqueNames.add("Alice");  // Duplicate ignored

// Check existence
boolean hasAlice = uniqueNames.contains("Alice");  // true

// Size
int size = uniqueNames.size();  // 2
```

**Characteristics:**
- Hash table implementation
- No ordering guarantee
- Fast operations (O(1) average)
- Allows one null element

### TreeSet
```java
Set<Integer> sortedNumbers = new TreeSet<>();
sortedNumbers.add(30);
sortedNumbers.add(10);
sortedNumbers.add(20);

// Automatically sorted
System.out.println(sortedNumbers);  // [10, 20, 30]

// Range operations
Integer higher = sortedNumbers.higher(20);  // 30
Integer lower = sortedNumbers.lower(20);    // 10
```

**Characteristics:**
- Red-black tree implementation
- Sorted natural ordering
- Slower operations (O(log n))
- No null elements

## Map Interface

### HashMap
```java
Map<String, Integer> ages = new HashMap<>();
ages.put("Alice", 25);
ages.put("Bob", 30);
ages.put("Charlie", 35);

// Access values
int aliceAge = ages.get("Alice");  // 25

// Check key existence
boolean hasBob = ages.containsKey("Bob");  // true

// Get all keys
Set<String> names = ages.keySet();
```

**Characteristics:**
- Hash table implementation
- No ordering guarantee
- Fast operations (O(1) average)
- Allows one null key and multiple null values

### TreeMap
```java
Map<String, Integer> sortedAges = new TreeMap<>();
sortedAges.put("Charlie", 35);
sortedAges.put("Alice", 25);
sortedAges.put("Bob", 30);

// Automatically sorted by key
System.out.println(sortedAges);  // {Alice=25, Bob=30, Charlie=35}

// Range operations
Map<String, Integer> subMap = sortedAges.headMap("Bob");  // {Alice=25}
```

**Characteristics:**
- Red-black tree implementation
- Sorted by keys
- Slower operations (O(log n))
- No null keys

## Queue Interface

### PriorityQueue
```java
Queue<Integer> priorityQueue = new PriorityQueue<>();
priorityQueue.add(30);
priorityQueue.add(10);
priorityQueue.add(20);

// Retrieves smallest element first
int smallest = priorityQueue.peek();  // 10
int removed = priorityQueue.poll();   // 10
```

### ArrayDeque (Stack replacement)
```java
Deque<Integer> stack = new ArrayDeque<>();
stack.push(10);   // Push to top
stack.push(20);
stack.push(30);

int top = stack.peek();  // 30
int popped = stack.pop(); // 30
```

## Iterating Collections

### For-each loop
```java
List<String> names = Arrays.asList("Alice", "Bob", "Charlie");
for (String name : names) {
    System.out.println(name);
}
```

### Iterator
```java
Iterator<String> iterator = names.iterator();
while (iterator.hasNext()) {
    String name = iterator.next();
    if (name.equals("Bob")) {
        iterator.remove();  // Safe removal
    }
}
```

### Java 8 Streams
```java
List<String> filtered = names.stream()
    .filter(name -> name.startsWith("A"))
    .collect(Collectors.toList());

List<Integer> lengths = names.stream()
    .map(String::length)
    .collect(Collectors.toList());
```

## Utility Methods

### Collections Class
```java
List<Integer> numbers = Arrays.asList(3, 1, 4, 1, 5);

// Sort
Collections.sort(numbers);

// Binary search (requires sorted list)
int index = Collections.binarySearch(numbers, 4);

// Shuffle
Collections.shuffle(numbers);

// Unmodifiable view
List<Integer> unmodifiable = Collections.unmodifiableList(numbers);

// Synchronized view
List<Integer> synchronized = Collections.synchronizedList(numbers);
```

## Choosing the Right Collection

| Use Case | Recommended Collection |
|----------|----------------------|
| Fast random access | ArrayList |
| Frequent insertion/deletion | LinkedList |
| Unique elements | HashSet |
| Sorted unique elements | TreeSet |
| Key-value mapping | HashMap |
| Sorted key-value mapping | TreeMap |
| Priority processing | PriorityQueue |
| Stack operations | ArrayDeque |

## Performance Considerations

### Time Complexity Summary
| Operation | ArrayList | LinkedList | HashSet | HashMap |
|-----------|-----------|------------|---------|---------|
| Add (end) | O(1)* | O(1) | O(1) | O(1) |
| Add (middle) | O(n) | O(1) | N/A | N/A |
| Remove (end) | O(1) | O(1) | O(1) | O(1) |
| Remove (middle) | O(n) | O(1) | O(1) | O(1) |
| Get by index | O(1) | O(n) | N/A | N/A |
| Contains | O(n) | O(n) | O(1) | O(1) |

*Amortized O(1) for ArrayList

## Best Practices

1. **Program to interfaces**: Use List, Set, Map as variable types
2. **Choose appropriate implementation**: Based on performance needs
3. **Initialize with proper capacity**: To avoid resizing
4. **Use generics**: For type safety
5. **Consider immutable collections**: For thread safety
6. **Use streams**: For functional-style operations
7. **Be aware of concurrency**: Use concurrent collections in multi-threaded code

The Collections Framework is essential for effective Java programming, providing powerful tools for data manipulation and storage.

Learn about Lists, Sets, Maps, and other collection interfaces for efficient data management

Java Collections Framework

# Java Programming

Java is a versatile, object-oriented programming language known for its platform independence and robust ecosystem.

## Overview

Java follows the "Write Once, Run Anywhere" philosophy, making it ideal for enterprise applications, Android development, and large-scale systems.

## Topics Covered

### Variables
Understanding primitive and reference types, memory allocation, and variable scoping in Java.

### Multithreading
Master concurrent programming with threads, synchronization, and modern concurrency utilities.

### Collections Framework
Learn about Lists, Sets, Maps, and other collection interfaces for efficient data management.

## Why Learn Java?

- **Platform Independence**: Run on any device with Java Virtual Machine
- **Strong Typing**: Catch errors at compile-time
- **Rich Ecosystem**: Extensive libraries and frameworks
- **Enterprise Ready**: Widely used in corporate environments
- **Android Development**: Primary language for Android apps

## Prerequisites

Basic understanding of programming concepts is helpful but not required. This path takes you from fundamentals to advanced topics.

Comprehensive guide to Java programming including variables, multithreading, and collections framework

Java Programming

# Python Programming

Python is a high-level, interpreted programming language known for its simplicity, readability, and extensive standard library.

## Why Python?

- **Simple Syntax**: Clean, readable code that's easy to learn
- **Versatile**: Web development, data science, AI, automation, and more
- **Large Ecosystem**: Extensive libraries and frameworks
- **Community Support**: Active community and abundant resources
- **Cross-Platform**: Runs on Windows, macOS, and Linux

## Key Features

### Dynamic Typing
```python
# No type declarations needed
name = "Python"
version = 3.9
is_awesome = True
```

### Interpreted Language
- No compilation step
- Interactive development
- Rapid prototyping

### Multi-Paradigm
- Object-oriented programming
- Functional programming
- Procedural programming

## Applications

### Web Development
- Django: Full-stack framework
- Flask: Lightweight framework
- FastAPI: Modern, high-performance framework

### Data Science & AI
- NumPy: Numerical computing
- Pandas: Data manipulation
- Scikit-learn: Machine learning
- TensorFlow/PyTorch: Deep learning

### Automation & Scripting
- System administration
- File operations
- Web scraping
- Task automation

## Getting Started

### Installation
```bash
# Download from python.org or use package manager
# macOS: brew install python3
# Ubuntu: sudo apt-get install python3
```

### First Program
```python
print("Hello, Python!")

# Variables and basic operations
name = "World"
age = 25
print(f"Hello, {name}! You are {age} years old.")
```

## Learning Path

1. **Basics**: Variables, data types, control flow
2. **Functions**: Reusable code blocks
3. **Data Structures**: Lists, dictionaries, sets, tuples
4. **Object-Oriented Programming**: Classes and objects
5. **Modules and Packages**: Code organization
6. **Error Handling**: Try-except blocks
7. **File Operations**: Reading and writing files

## Popular Libraries

### Web Development
- **Requests**: HTTP library
- **Beautiful Soup**: HTML parsing
- **Selenium**: Web automation

### Data Science
- **Matplotlib**: Plotting
- **Seaborn**: Statistical visualization
- **Jupyter**: Interactive notebooks

### General Purpose
- **Os**: Operating system interface
- **Sys**: System-specific parameters
- **Json**: JSON data handling
- **Datetime**: Date and time manipulation

Python's simplicity and power make it an excellent choice for both beginners and experienced developers.

Learn Python programming from basics to advanced concepts with practical examples

Python Programming

# JavaScript Programming

JavaScript is a versatile, high-level programming language primarily known for powering web applications but now used across multiple platforms.

## Why JavaScript?

- **Ubiquitous**: Runs in every web browser
- **Full-Stack**: Frontend and backend development
- **Event-Driven**: Perfect for interactive applications
- **Large Ecosystem**: npm, frameworks, and tools
- **Community**: Massive developer community

## Key Features

### Dynamic Typing
```javascript
let name = "JavaScript";  // String
let version = 2023;      // Number
let isAwesome = true;     // Boolean
```

### First-Class Functions
```javascript
// Functions as values
const greet = function(name) {
    return `Hello, ${name}!`;
};

// Functions as arguments
function execute(func, arg) {
    return func(arg);
}

execute(greet, "World");  // "Hello, World!"
```

### Prototypal Inheritance
```javascript
const person = {
    name: "John",
    greet() {
        return `Hello, I'm ${this.name}`;
    }
};

const student = Object.create(person);
student.study = function() {
    return `${this.name} is studying`;
};
```

## Applications

### Frontend Development
- **React**: Component-based UI library
- **Vue**: Progressive framework
- **Angular**: Full-featured framework
- **Vanilla JS**: No framework approach

### Backend Development
- **Node.js**: JavaScript runtime
- **Express**: Web framework
- **NestJS**: TypeScript framework
- **Fastify**: High-performance server

### Mobile Development
- **React Native**: Cross-platform mobile apps
- **Ionic**: Hybrid mobile apps
- **NativeScript**: Native mobile apps

### Desktop Applications
- **Electron**: Cross-platform desktop apps
- **Tauri**: Lightweight alternative to Electron

## Modern JavaScript (ES6+)

### Arrow Functions
```javascript
const add = (a, b) => a + b;
const greet = name => `Hello, ${name}!`;
const square = x => x * x;
```

### Destructuring
```javascript
// Array destructuring
const [first, second] = [1, 2, 3];

// Object destructuring
const {name, age} = {name: "Alice", age: 25};
```

### Template Literals
```javascript
const name = "World";
const message = `Hello, ${name}!
Welcome to JavaScript.`;
```

### Async/Await
```javascript
async function fetchData() {
    try {
        const response = await fetch('/api/data');
        const data = await response.json();
        return data;
    } catch (error) {
        console.error('Error:', error);
    }
}
```

## Core Concepts

### Variables and Scope
```javascript
// Global scope
var globalVar = "global";

// Function scope
function example() {
    var functionVar = "function";
    
    if (true) {
        let blockVar = "block";    // Block scope
        const constant = "immutable";
    }
}
```

### Closures
```javascript
function counter() {
    let count = 0;
    
    return function() {
        count++;
        return count;
    };
}

const increment = counter();
console.log(increment()); // 1
console.log(increment()); // 2
```

### Promises
```javascript
const promise = new Promise((resolve, reject) => {
    setTimeout(() => {
        resolve("Success!");
    }, 1000);
});

promise.then(result => {
    console.log(result);
}).catch(error => {
    console.error(error);
});
```

## Popular Libraries and Frameworks

### Frontend Frameworks
- **React**: Component-based, virtual DOM
- **Vue**: Progressive, easy to learn
- **Angular**: Full-featured, TypeScript-based

### Backend Frameworks
- **Express**: Minimalist, flexible
- **Koa**: Modern, middleware-focused
- **Fastify**: High-performance, plugin-based

### Utility Libraries
- **Lodash**: Functional programming utilities
- **Axios**: HTTP client
- **Moment.js**: Date manipulation (consider alternatives)

### Testing
- **Jest**: JavaScript testing framework
- **Mocha**: Feature-rich testing framework
- **Cypress**: End-to-end testing

## Development Tools

### Package Managers
- **npm**: Node Package Manager
- **Yarn**: Fast, reliable package manager
- **pnpm**: Fast, disk space efficient

### Build Tools
- **Webpack**: Module bundler
- **Vite**: Fast build tool
- **Rollup**: Module bundler for libraries

### Code Quality
- **ESLint**: JavaScript linter
- **Prettier**: Code formatter
- **TypeScript**: Static typing

## Best Practices

1. **Use const and let**: Avoid var
2. **Write modular code**: Use modules and classes
3. **Handle errors**: Try-catch and error boundaries
4. **Optimize performance**: Lazy loading, code splitting
5. **Use TypeScript**: For large applications
6. **Test thoroughly**: Unit tests, integration tests
7. **Follow naming conventions**: Consistent and descriptive names

JavaScript's versatility and ecosystem make it an essential language for modern web development and beyond.

Master JavaScript for web development and beyond with modern ES6+ features

JavaScript Programming

# Data Structures

Data structures are fundamental building blocks for organizing and storing data efficiently. Understanding data structures is crucial for writing efficient algorithms and solving complex problems.

## Why Study Data Structures?

- **Efficiency**: Choose the right structure for optimal performance
- **Problem Solving**: Different structures solve different problems
- **Interview Preparation**: Essential for technical interviews
- **Software Design**: Foundation of good software architecture

## Fundamental Data Structures

### Arrays
Contiguous memory storage with O(1) random access.

**Use Cases:**
- Storing lists of similar items
- Implementing other data structures
- Matrix operations
- Buffer management

**Time Complexity:**
- Access: O(1)
- Search: O(n)
- Insertion (end): O(1)
- Insertion (middle): O(n)

### Linked Lists
Nodes with data and pointers, non-contiguous memory.

**Types:**
- **Singly Linked**: One-way traversal
- **Doubly Linked**: Two-way traversal
- **Circular**: Last node points to first

**Use Cases:**
- Implementing stacks and queues
- Browser history
- Music playlists
- Dynamic memory allocation

### Stacks
LIFO (Last In, First Out) structure.

**Operations:**
- Push: Add element to top
- Pop: Remove element from top
- Peek: View top element

**Use Cases:**
- Function call stack
- Expression evaluation
- Undo/redo functionality
- Browser back button

### Queues
FIFO (First In, First Out) structure.

**Operations:**
- Enqueue: Add element to rear
- Dequeue: Remove element from front
- Front: View front element

**Use Cases:**
- Task scheduling
- Print queue management
- Message queues
- Breadth-first search

### Trees
Hierarchical structure with parent-child relationships.

**Types:**
- **Binary Trees**: Each node has at most 2 children
- **Binary Search Trees**: Ordered binary trees
- **AVL Trees**: Self-balancing binary trees
- **B-Trees**: Multi-way search trees

**Use Cases:**
- File systems
- Database indexes
- DOM trees
- Decision trees

### Graphs
Nodes connected by edges, representing relationships.

**Types:**
- **Directed**: Edges have direction
- **Undirected**: Edges are bidirectional
- **Weighted**: Edges have weights
- **Unweighted**: All edges equal

**Use Cases:**
- Social networks
- Maps and navigation
- Network routing
- Dependency graphs

### Hash Tables
Key-value pairs with O(1) average case operations.

**Components:**
- **Hash Function**: Maps keys to array indices
- **Array**: Storage for values
- **Collision Resolution**: Handling key conflicts

**Use Cases:**
- Dictionaries and maps
- Caching mechanisms
- Database indexing
- Symbol tables

## Advanced Data Structures

### Heaps
Specialized tree-based structure satisfying heap property.

**Types:**
- **Min-Heap**: Parent ≤ children
- **Max-Heap**: Parent ≥ children

**Use Cases:**
- Priority queues
- Heap sort
- Finding kth largest/smallest element
- Dijkstra's algorithm

### Tries
Prefix tree for efficient string operations.

**Use Cases:**
- Autocomplete
- Spell checking
- IP routing tables
- Dictionary implementations

### Disjoint Set Union
Tracks partition of elements into disjoint sets.

**Operations:**
- Find: Determine which set element belongs to
- Union: Merge two sets
- MakeSet: Create new set

**Use Cases:**
- Kruskal's algorithm
- Connected components
- Dynamic connectivity

## Choosing the Right Data Structure

| Requirement | Recommended Structure |
|-------------|----------------------|
| Fast random access | Array |
| Dynamic size | Linked List |
| LIFO behavior | Stack |
| FIFO behavior | Queue |
| Hierarchical data | Tree |
| Key-value lookup | Hash Table |
| Network relationships | Graph |
| Priority processing | Heap |
| String operations | Trie |

## Time Complexity Summary

| Structure | Access | Search | Insertion | Deletion |
|-----------|--------|--------|-----------|----------|
| Array | O(1) | O(n) | O(1)* | O(n) |
| Linked List | O(n) | O(n) | O(1) | O(1) |
| Stack | O(n) | O(n) | O(1) | O(1) |
| Queue | O(n) | O(n) | O(1) | O(1) |
| Binary Search Tree | O(log n) | O(log n) | O(log n) | O(log n) |
| Hash Table | N/A | O(1) | O(1) | O(1) |
| Heap | O(n) | O(n) | O(log n) | O(log n) |

*Amortized O(1) for dynamic arrays

## Implementation Tips

### Memory Management
- Choose appropriate initial sizes
- Handle resizing efficiently
- Consider memory fragmentation

### Performance Optimization
- Profile before optimizing
- Consider cache locality
- Minimize pointer chasing
- Use appropriate data types

### Error Handling
- Validate input parameters
- Handle edge cases
- Manage memory allocation failures
- Provide clear error messages

## Common Algorithms

### Sorting
- **Quick Sort**: O(n log n) average, O(n²) worst
- **Merge Sort**: O(n log n) stable
- **Heap Sort**: O(n log n) in-place
- **Counting Sort**: O(n) for integers

### Searching
- **Binary Search**: O(log n) for sorted arrays
- **Depth-First Search**: For trees and graphs
- **Breadth-First Search**: For shortest path
- **Hash-based Search**: O(1) average

### Traversal
- **Inorder, Preorder, Postorder**: Tree traversals
- **Level Order**: Breadth-first tree traversal
- **Topological Sort**: Directed acyclic graphs

## Best Practices

1. **Understand Requirements**: Choose structure based on use case
2. **Consider Complexity**: Analyze time and space complexity
3. **Test Thoroughly**: Test with various input sizes
4. **Document Assumptions**: Clearly state constraints
5. **Reuse Standard Implementations**: Use well-tested libraries
6. **Profile Real Data**: Test with actual usage patterns

Mastering data structures provides the foundation for efficient algorithm design and problem-solving in software development.

Understanding fundamental data structures for efficient programming and problem solving

Data Structures

# Foundations

The foundations layer covers fundamental programming concepts and languages that form the building blocks of software development.

## Topics

### Java

Comprehensive guide to Java programming including variables, multithreading, and collections framework.

### Python

Learn Python programming from basics to advanced concepts.

### JavaScript

Master JavaScript for web development and beyond.

### Data Structures

Understanding fundamental data structures for efficient programming.

## Learning Path

Start with any programming language of your choice, then explore data structures to strengthen your problem-solving skills. Each topic builds upon fundamental concepts while maintaining practical applicability.

Fundamental programming concepts and languages that form the building blocks of software development

Foundations

# Java Primitive Types

Java provides eight primitive data types for basic values. These are the building blocks of Java programming.

## The Eight Primitive Types

### Integer Types

#### byte
- **Size**: 8 bits (1 byte)
- **Range**: -128 to 127
- **Default**: 0
- **Use Case**: Saving memory in large arrays

```java
byte smallNumber = 100;
byte temperature = -20;
```

#### short
- **Size**: 16 bits (2 bytes)
- **Range**: -32,768 to 32,767
- **Default**: 0
- **Use Case**: Memory-efficient integer storage

```java
short year = 2023;
short port = 8080;
```

#### int
- **Size**: 32 bits (4 bytes)
- **Range**: -2³¹ to 2³¹-1
- **Default**: 0
- **Use Case**: General-purpose integer

```java
int count = 42;
int population = 1_000_000;
```

#### long
- **Size**: 64 bits (8 bytes)
- **Range**: -2⁶³ to 2⁶³-1
- **Default**: 0L
- **Use Case**: Large numbers, timestamps

```java
long bigNumber = 9_000_000_000L;
long timestamp = System.currentTimeMillis();
```

### Floating-Point Types

#### float
- **Size**: 32 bits (4 bytes)
- **Precision**: 6-7 decimal digits
- **Default**: 0.0f
- **Use Case**: Memory-efficient decimal numbers

```java
float pi = 3.14159f;
float price = 19.99f;
```

#### double
- **Size**: 64 bits (8 bytes)
- **Precision**: 15-16 decimal digits
- **Default**: 0.0
- **Use Case**: Precise decimal calculations

```java
double precisePi = 3.141592653589793;
double balance = 1234.56;
```

### Other Types

#### boolean
- **Size**: 1 bit (implementation dependent)
- **Values**: true or false
- **Default**: false
- **Use Case**: Conditional logic

```java
isActive = true;
hasPermission = false;
```

#### char
- **Size**: 16 bits (2 bytes)
- **Range**: '\u0000' to '\uffff' (Unicode)
- **Default**: '\u0000'
- **Use Case**: Single characters

```java
char grade = 'A';
char newline = '\n';
char unicode = '\u03A9'; // Greek omega
```

## Type Conversion

### Implicit Conversion (Widening)
```java
int intValue = 100;
long longValue = intValue;  // int to long (automatic)
double doubleValue = intValue;  // int to double (automatic)
```

### Explicit Conversion (Narrowing)
```java
double doubleValue = 123.456;
int intValue = (int) doubleValue;  // 123 (truncates decimal)
long longValue = (long) doubleValue;  // 123L
```

## Default Values
When primitive variables are declared as class members without initialization:

```java
public class Defaults {
    byte b;      // 0
    short s;     // 0
    int i;       // 0
    long l;      // 0L
    float f;     // 0.0f
    double d;    // 0.0
    boolean bool; // false
    char c;      // '\u0000'
}
```

## Best Practices

1. **Choose the smallest type** that fits your needs to save memory
2. **Use long for timestamps** and large numbers
3. **Prefer double over float** for precision
4. **Use underscores** for readability in large numbers
5. **Be explicit with literals** (f for float, L for long)

## Common Pitfalls

- **Integer overflow**: `int max = Integer.MAX_VALUE + 1;` // becomes negative
- **Floating-point precision**: `0.1 + 0.2 != 0.3` due to binary representation
- **Division by zero**: Throws ArithmeticException for integers
- **Type casting loss**: `(int) 3.99` results in 3, not 4

Understanding primitive types is fundamental to writing efficient and correct Java code.

Understanding Java's eight primitive data types and their characteristics

Java Primitive Types

# Java Reference Types

Reference types in Java store addresses that point to objects in memory. Unlike primitive types, they don't hold the actual data but references to where the data is stored.

## Types of Reference Variables

### Objects
Instances of classes that contain both data (fields) and behavior (methods).

```java
// Creating objects
String name = new String("Java Developer");
Date today = new Date();
ArrayList<String> list = new ArrayList<>();

// Object references point to memory locations
String str1 = "Hello";
String str2 = str1;  // Both reference the same object
```

### Arrays
Collections of elements of the same type stored in contiguous memory.

```java
// Array declarations
int[] numbers = new int[5];
String[] names = new String[3];
int[][] matrix = new int[3][3];  // 2D array

// Array initialization
int[] primes = {2, 3, 5, 7, 11};
String[] days = {"Mon", "Tue", "Wed", "Thu", "Fri"};
```

### Strings
Special objects for text manipulation with unique memory handling.

```java
// String literals (stored in string pool)
String literal1 = "Hello";
String literal2 = "Hello";  // Same object as literal1

// String objects (stored in heap)
String object1 = new String("Hello");
String object2 = new String("Hello");  // Different object
```

## Memory Management

### Stack vs Heap

#### Stack Memory
- Stores primitive variables and reference variables
- Fast access
- Limited size
- Automatically managed (LIFO order)

```java
public void method() {
    int x = 10;              // Stack
    String name = "Java";    // Reference on stack, object on heap
    int[] numbers = new int[5]; // Reference on stack, array on heap
}
```

#### Heap Memory
- Stores objects and arrays
- Larger size
- Slower access than stack
- Managed by garbage collector

```java
// These objects are created in heap memory
Person person = new Person("Alice", 25);
List<Integer> list = new ArrayList<>();
int[] largeArray = new int[1000000];
```

## Object Lifecycle

### Creation
```java
// 1. Declaration (reference on stack)
Person person;

// 2. Instantiation (object on heap)
person = new Person("Alice", 25);

// 3. Initialization
person.setName("Alice Smith");
```

### Usage
```java
// Accessing object members
String name = person.getName();
person.setAge(26);

// Method calls
person.birthday();  // Increments age
```

### Garbage Collection
```java
public void createObjects() {
    Person person1 = new Person("Alice");
    Person person2 = new Person("Bob");
    
    person1 = null;  // Eligible for garbage collection
    // person2 still referenced
} // person2 becomes eligible at method end
```

## Reference Types in Detail

### Class Objects
```java
public class Car {
    private String model;
    private int year;
    
    public Car(String model, int year) {
        this.model = model;
        this.year = year;
    }
    
    // Methods...
}

// Usage
Car myCar = new Car("Toyota", 2023);
Car yourCar = myCar;  // Both reference same object
```

### Interface References
```java
// Interface reference pointing to implementation
List<String> list = new ArrayList<>();
Map<String, Integer> map = new HashMap<>();
Runnable task = new MyTask();
```

### Abstract Class References
```java
// Abstract class reference pointing to concrete implementation
Animal dog = new Dog();
Animal cat = new Cat();
```

## Special Reference Types

### Null Reference
```java
String text = null;  // Reference to nothing
if (text != null) {
    System.out.println(text.length());
}
```

### This Reference
```java
public class Person {
    private String name;
    
    public void setName(String name) {
        this.name = name;  // 'this' refers to current object
    }
}
```

### Super Reference
```java
public class Student extends Person {
    public void displayInfo() {
        super.displayInfo();  // Calls parent method
        System.out.println("Student info");
    }
}
```

## Pass by Value vs Pass by Reference

Java is always pass-by-value, but with reference types, the value passed is the reference.

```java
public void modifyString(String str) {
    str = str + " Modified";  // Creates new string, doesn't change original
}

public void modifyList(List<String> list) {
    list.add("New Item");  // Modifies the object referenced
}

String original = "Hello";
modifyString(original);
System.out.println(original);  // Still "Hello"

List<String> items = new ArrayList<>();
items.add("Original");
modifyList(items);
System.out.println(items);  // Contains "Original" and "New Item"
```

## Comparison of Reference Types

### Equality (== vs equals)
```java
String str1 = new String("Hello");
String str2 = new String("Hello");
String str3 = str1;

System.out.println(str1 == str2);    // false (different objects)
System.out.println(str1.equals(str2)); // true (same content)
System.out.println(str1 == str3);    // true (same object)
```

### Arrays Comparison
```java
int[] arr1 = {1, 2, 3};
int[] arr2 = {1, 2, 3};
int[] arr3 = arr1;

System.out.println(arr1 == arr2);    // false (different arrays)
System.out.println(Arrays.equals(arr1, arr2)); // true (same contents)
System.out.println(arr1 == arr3);    // true (same array)
```

## Memory Optimization

### Object Reuse
```java
// Good: Reuse objects
StringBuilder builder = new StringBuilder();
for (int i = 0; i < 1000; i++) {
    builder.setLength(0);  // Clear and reuse
    builder.append("Item ").append(i);
    process(builder.toString());
}

// Bad: Creating many objects
for (int i = 0; i < 1000; i++) {
    String str = "Item " + i;  // Creates new string each iteration
    process(str);
}
```

### Weak References
```java
import java.lang.ref.WeakReference;

WeakReference<Person> weakRef = new WeakReference<>(new Person("Temp"));
Person person = weakRef.get();  // May return null if garbage collected
```

## Best Practices

1. **Initialize references** to avoid NullPointerException
2. **Use null checks** before accessing object members
3. **Prefer interfaces** for reference types when possible
4. **Be aware of object creation** in loops
5. **Use appropriate collection types** based on usage patterns
6. **Consider memory usage** when creating large objects

## Common Pitfalls

- **NullPointerException**: Accessing null reference
- **Memory leaks**: Holding references to unused objects
- **Unnecessary object creation**: In loops or frequent methods
- **String concatenation**: Using + in loops creates many objects
- **Array vs ArrayList**: Fixed vs dynamic size considerations

Understanding reference types and memory management is crucial for writing efficient and robust Java applications.

Understanding Java's reference types and object memory management

Java Reference Types

# Java Memory Allocation

Java memory allocation is a fundamental concept that determines where and how data is stored during program execution. Understanding memory allocation helps write efficient and bug-free code.

## Memory Areas in Java

### 1. Stack Memory
Stack memory is used for static memory allocation and stores:
- Primitive variables
- Reference variables (addresses)
- Method call information
- Local variables

**Characteristics:**
- LIFO (Last In, First Out) structure
- Fast access
- Limited size (typically 1-2MB per thread)
- Automatically managed

```java
public void calculate() {
    int x = 10;                    // Stack
    String name = "Java";          // Reference on stack, object on heap
    int[] numbers = new int[5];    // Reference on stack, array on heap
    
    if (x > 5) {
        int y = 20;                // Stack (local to if block)
        String temp = name;        // Stack reference
    } // y and temp popped from stack here
} // x, name, numbers popped from stack here
```

### 2. Heap Memory
Heap memory is used for dynamic memory allocation and stores:
- Objects
- Arrays
- Instance variables

**Characteristics:**
- Larger than stack
- Slower access than stack
- Managed by garbage collector
- Shared across all threads

```java
public class Example {
    private int instanceVar;  // Heap (part of object)
    
    public void method() {
        Example obj = new Example();  // Object on heap
        int[] arr = new int[1000];   // Array on heap
    }
}
```

### 3. Method Area
Stores:
- Class-level information
- Static variables
- Method bytecode
- Constant pool

```java
public class Constants {
    public static final int MAX_SIZE = 100;  // Method area
    private static String name = "Java";      // Method area
    
    public static void staticMethod() {       // Method bytecode in method area
        // Method implementation
    }
}
```

### 4. Program Counter (PC Register
Stores:
- Current instruction address
- Thread-specific

## Variable Allocation Examples

### Primitive Variables
```java
public class PrimitiveAllocation {
    public static void main(String[] args) {
        // All primitives on stack
        byte b = 10;        // 1 byte on stack
        short s = 20;       // 2 bytes on stack
        int i = 30;         // 4 bytes on stack
        long l = 40L;       // 8 bytes on stack
        float f = 50.5f;    // 4 bytes on stack
        double d = 60.6;    // 8 bytes on stack
        boolean bool = true; // 1 bit on stack
        char c = 'A';        // 2 bytes on stack
    }
}
```

### Reference Variables
```java
public class ReferenceAllocation {
    public static void main(String[] args) {
        // References on stack, objects on heap
        String str = "Hello";           // Reference on stack, literal in pool
        String obj = new String("World"); // Reference on stack, object on heap
        
        int[] numbers = new int[5];     // Reference on stack, array on heap
        Person person = new Person();   // Reference on stack, object on heap
    }
}
```

### Object Memory Layout
```java
public class Person {
    private String name;    // Reference (4-8 bytes)
    private int age;        // Primitive (4 bytes)
    private boolean active; // Primitive (1 byte)
    // + Object header (typically 12-16 bytes)
    // + Padding for alignment
}

// Memory usage:
// Object header: ~12 bytes
// Reference: 8 bytes (64-bit JVM)
// int: 4 bytes
// boolean: 1 byte
// Padding: 3 bytes (for 8-byte alignment)
// Total: ~28 bytes per Person object
```

## Array Memory Allocation

### One-Dimensional Arrays
```java
public class ArrayAllocation {
    public static void main(String[] args) {
        int[] numbers = new int[5];
        // Array object on heap: 20 bytes (5 * 4 bytes)
        // Reference on stack: 8 bytes
        
        String[] names = new String[3];
        // Array object on heap: 24 bytes (3 * 8 bytes for references)
        // References initially null
        
        names[0] = "Alice";  // String literal in string pool
        names[1] = new String("Bob"); // String object on heap
        names[2] = "Charlie"; // String literal in string pool
    }
}
```

### Multi-Dimensional Arrays
```java
public class MultiArrayAllocation {
    public static void main(String[] args) {
        int[][] matrix = new int[3][4];
        // Outer array on heap: 24 bytes (3 * 8 bytes for references)
        // 3 inner arrays on heap: 3 * 16 bytes (4 * 4 bytes each)
        // Reference on stack: 8 bytes
        
        // Ragged arrays
        int[][] ragged = new int[3][];
        ragged[0] = new int[2];  // 8 bytes
        ragged[1] = new int[4];  // 16 bytes
        ragged[2] = new int[3];  // 12 bytes
    }
}
```

## String Memory Allocation

### String Literal Pool
```java
public class StringAllocation {
    public static void main(String[] args) {
        String s1 = "Hello";        // String pool
        String s2 = "Hello";        // Same object as s1
        String s3 = new String("Hello"); // Heap
        
        System.out.println(s1 == s2);    // true (same object)
        System.out.println(s1 == s3);    // false (different objects)
    }
}
```

### String Concatenation
```java
public class StringConcatenation {
    public static void main(String[] args) {
        // Compile-time concatenation
        String s1 = "Hello" + " World";  // Single string in pool
        
        // Runtime concatenation
        String s2 = "Hello";
        String s3 = s2 + " World";       // Creates new StringBuilder object
        
        // Efficient concatenation in loops
        StringBuilder sb = new StringBuilder();
        for (int i = 0; i < 100; i++) {
            sb.append("Item ").append(i).append("\n");
        }
        String result = sb.toString();
    }
}
```

## Garbage Collection

### When Objects Become Eligible for GC
```java
public class GarbageCollection {
    public static void method1() {
        Person person = new Person("Alice");
        person = null;  // Eligible for GC
    }
    
    public static void method2() {
        Person person1 = new Person("Alice");
        Person person2 = new Person("Bob");
        person1 = person2;  // Original person1 object eligible for GC
    }
    
    public static void method3() {
        createPerson();  // Person object eligible for GC after method returns
    }
    
    private static Person createPerson() {
        return new Person("Alice");
    }
}
```

### Memory Leaks
```java
public class MemoryLeaks {
    private static List<Person> cache = new ArrayList<>();
    
    // Memory leak: Objects never removed
    public void addToCache(Person person) {
        cache.add(person);  // Objects remain in cache forever
    }
    
    // Fixed: Use weak references or explicit cleanup
    public void properCacheManagement() {
        // Use WeakReference for automatic cleanup
        Map<String, WeakReference<Person>> weakCache = new HashMap<>();
        
        // Or implement explicit cleanup
        if (cache.size() > MAX_CACHE_SIZE) {
            cache.clear();  // Explicit cleanup
        }
    }
}
```

## Memory Optimization Techniques

### Object Pooling
```java
public class ObjectPool {
    private static final List<Person> pool = new ArrayList<>();
    
    public static Person getPerson() {
        if (pool.isEmpty()) {
            return new Person();
        }
        return pool.remove(pool.size() - 1);
    }
    
    public static void returnPerson(Person person) {
        person.reset();  // Reset state
        pool.add(person);
    }
}
```

### Primitive vs Object Choice
```java
public class Optimization {
    // Good: Use primitives when possible
    private int count;           // 4 bytes
    private double price;        // 8 bytes
    
    // Avoid: Unnecessary object creation
    private Integer countObj;    // Reference + object overhead
    
    // Use primitive collections for large datasets
    private IntArrayList intList;  // Specialized primitive collection
    private ArrayList<Integer> integerList;  // Object overhead
}
```

## Memory Analysis Tools

### JVM Memory Flags
```bash
# Heap size settings
-Xms512m                    # Initial heap size
-Xmx2g                      # Maximum heap size
-XX:NewSize=128m            # Young generation size
-XX:MaxNewSize=512m         # Maximum young generation size

# Garbage collection
-XX:+UseG1GC                # Use G1 garbage collector
-XX:+PrintGC                # Print GC information
-XX:+PrintGCDetails         # Detailed GC information
```

### Memory Profiling
```java
public class MemoryProfiling {
    public static void main(String[] args) {
        Runtime runtime = Runtime.getRuntime();
        
        long totalMemory = runtime.totalMemory();
        long freeMemory = runtime.freeMemory();
        long usedMemory = totalMemory - freeMemory;
        long maxMemory = runtime.maxMemory();
        
        System.out.println("Used Memory: " + (usedMemory / 1024 / 1024) + " MB");
        System.out.println("Free Memory: " + (freeMemory / 1024 / 1024) + " MB");
        System.out.println("Total Memory: " + (totalMemory / 1024 / 1024) + " MB");
        System.out.println("Max Memory: " + (maxMemory / 1024 / 1024) + " MB");
    }
}
```

## Best Practices

1. **Use primitives** when possible to avoid object overhead
2. **Initialize objects** properly to avoid null references
3. **Be aware of string concatenation** in loops
4. **Use appropriate collections** based on usage patterns
5. **Implement proper cleanup** for resources
6. **Monitor memory usage** in production
7. **Use object pooling** for frequently created/destroyed objects

## Common Memory Issues

- **OutOfMemoryError**: Heap exhausted
- **StackOverflowError**: Stack space exhausted
- **Memory leaks**: Objects not garbage collected
- **Excessive object creation**: Performance degradation
- **Large arrays**: Memory fragmentation

Understanding Java memory allocation is essential for writing efficient, scalable applications and troubleshooting memory-related issues.

Understanding how Java allocates and manages memory for variables and objects

Java Memory Allocation

# Java Variable Scoping

Variable scope determines where a variable can be accessed in your code. Understanding scope is crucial for writing clean, maintainable, and bug-free Java programs.

## Types of Variable Scope

### 1. Class/Instance Scope
Variables declared at the class level, outside any method.

#### Instance Variables
- Belong to instances of the class
- Each object has its own copy
- Accessible throughout the class
- Lifetime: As long as the object exists

```java
public class Employee {
    private String name;        // Instance variable
    private int age;            // Instance variable
    protected double salary;    // Instance variable (protected access)
    public boolean isActive;    // Instance variable (public access)
    
    public void displayInfo() {
        System.out.println(name);  // Can access instance variables
        System.out.println(age);
    }
    
    public void setAge(int age) {
        this.age = age;  // 'this' distinguishes from parameter
    }
}

// Usage
Employee emp1 = new Employee();
Employee emp2 = new Employee();
emp1.name = "Alice";  // emp1.name is different from emp2.name
emp2.name = "Bob";
```

#### Static Variables
- Belong to the class, not instances
- Shared among all objects
- Accessed using class name
- Lifetime: As long as the class is loaded

```java
public class Counter {
    private static int count = 0;        // Static variable
    private int instanceCount = 0;       // Instance variable
    
    public Counter() {
        count++;           // Shared among all instances
        instanceCount++;  // Specific to this instance
    }
    
    public static int getCount() {
        return count;      // Can access static variables
    }
    
    public int getInstanceCount() {
        return instanceCount;
    }
}

// Usage
Counter c1 = new Counter();  // count = 1, c1.instanceCount = 1
Counter c2 = new Counter();  // count = 2, c2.instanceCount = 1
System.out.println(Counter.getCount());  // 2 (shared)
```

### 2. Method Scope
Variables declared inside methods.

#### Local Variables
- Declared inside methods or constructors
- Only accessible within the method
- Lifetime: From declaration to method completion
- Must be initialized before use

```java
public class Calculator {
    public int add(int a, int b) {
        int result = a + b;  // Local variable
        return result;
    }
    
    public void processNumbers() {
        int x = 10;          // Local variable
        int y = 20;          // Local variable
        
        if (x > 5) {
            int sum = x + y; // Local to if block
            System.out.println(sum);
        }
        
        // sum is not accessible here
        // System.out.println(sum);  // Compile error
    }
}
```

#### Method Parameters
- Special local variables that receive method arguments
- Accessible throughout the method

```java
public class Greeter {
    public void greet(String name, int times) {  // Parameters
        for (int i = 0; i < times; i++) {      // i is local to loop
            System.out.println("Hello, " + name);
        }
        // name and times accessible throughout method
        // i not accessible here
    }
}
```

### 3. Block Scope
Variables declared within specific code blocks.

#### If-Else Blocks
```java
public class BlockScope {
    public void checkNumber(int number) {
        if (number > 0) {
            String message = "Positive";  // Block scope
            System.out.println(message);
        } else if (number < 0) {
            String message = "Negative";  // Different variable
            System.out.println(message);
        } else {
            String message = "Zero";        // Different variable
            System.out.println(message);
        }
        
        // message not accessible here
    }
}
```

#### Loop Blocks
```java
public class LoopScope {
    public void demonstrateLoops() {
        for (int i = 0; i < 5; i++) {
            int temp = i * 2;  // Local to loop
            System.out.println(temp);
        }
        // i and temp not accessible here
        
        int j = 0;
        while (j < 3) {
            int temp = j * 3;  // Different from temp in for loop
            System.out.println(temp);
            j++;
        }
    }
}
```

#### Try-Catch Blocks
```java
public class ExceptionScope {
    public void handleException() {
        try {
            String data = readFile();  // Local to try block
            System.out.println(data);
        } catch (IOException e) {
            String error = "Error: " + e.getMessage();  // Local to catch block
            System.out.println(error);
        } finally {
            // Neither data nor error accessible here
            System.out.println("Cleanup completed");
        }
    }
}
```

## Variable Shadowing

### Parameter Shadowing
When a parameter has the same name as an instance variable.

```java
public class ShadowingExample {
    private int value = 10;  // Instance variable
    
    public void setValue(int value) {  // Parameter shadows instance variable
        this.value = value;  // 'this' refers to instance variable
        // Without 'this', 'value' refers to parameter
    }
    
    public void printValue() {
        System.out.println(value);  // Instance variable
    }
}
```

### Local Variable Shadowing
When a local variable shadows an instance variable.

```java
public class LocalShadowing {
    private String message = "Instance message";
    
    public void demonstrate() {
        String message = "Local message";  // Shadows instance variable
        
        System.out.println(message);        // Local variable
        System.out.println(this.message);   // Instance variable
    }
}
```

## Access Modifiers and Scope

### Private
- Only accessible within the same class
- Most restrictive access level

```java
public class PrivateExample {
    private int privateVar = 10;
    
    private void privateMethod() {
        // Only accessible within this class
    }
    
    public void publicMethod() {
        privateVar = 20;      // Can access private members
        privateMethod();       // Can call private methods
    }
}
```

### Default (Package-Private)
- Accessible within the same package
- No modifier needed

```java
class PackageExample {  // Default access
    int packageVar = 10;  // Default access
    
    void packageMethod() {  // Default access
        // Accessible within same package
    }
}
```

### Protected
- Accessible within same package and subclasses
- More permissive than default

```java
public class ProtectedExample {
    protected int protectedVar = 10;
    
    protected void protectedMethod() {
        // Accessible in same package and subclasses
    }
}

public class Subclass extends ProtectedExample {
    public void useProtected() {
        protectedVar = 20;      // Can access protected member
        protectedMethod();       // Can call protected method
    }
}
```

### Public
- Accessible from anywhere
- Most permissive access level

```java
public class PublicExample {
    public int publicVar = 10;
    
    public void publicMethod() {
        // Accessible from any class
    }
}
```

## Scope and Memory Management

### Stack Frame Management
```java
public class StackManagement {
    public static void methodA() {
        int a = 10;  // Pushed to stack when methodA called
        
        methodB();   // methodB's frame pushed to stack
        
        int b = 20;  // Still accessible after methodB returns
    }
    
    public static void methodB() {
        int c = 30;  // Pushed to stack
        // c is popped when methodB returns
    }
}
```

### Variable Lifetime Examples
```java
public class LifetimeExample {
    private static int staticVar = 1;      // Lifetime: Class loading to JVM shutdown
    private int instanceVar = 2;           // Lifetime: Object creation to garbage collection
    
    public void method() {
        int localVar = 3;                   // Lifetime: Method entry to exit
        
        if (localVar > 0) {
            int blockVar = 4;              // Lifetime: Block entry to exit
        }
        
        for (int i = 0; i < 5; i++) {      // i: Loop iteration start to end
            int loopVar = i * 2;            // loopVar: Each iteration
        }
    }
}
```

## Best Practices

### 1. Use the Most Restrictive Scope
```java
public class GoodScope {
    private int instanceVar;  // Instance scope when needed across methods
    
    public void method() {
        int localVar;        // Local scope when only needed in method
        
        if (condition) {
            int blockVar;   // Block scope when only needed in block
        }
    }
}
```

### 2. Initialize Variables Appropriately
```java
public class Initialization {
    private int instanceVar = 0;        // Instance variables often initialized
    private static int staticVar = 0;    // Static variables should be initialized
    
    public void method() {
        int localVar;                   // Local variables must be initialized before use
        // System.out.println(localVar); // Compile error
        
        int initializedVar = 10;        // Proper initialization
        System.out.println(initializedVar);
    }
}
```

### 3. Avoid Variable Shadowing When Possible
```java
public class AvoidShadowing {
    private int employeeId;
    
    public void setEmployeeId(int id) {  // Different name from instance variable
        this.employeeId = id;           // Clear distinction
    }
    
    // Instead of:
    public void setEmployeeId(int employeeId) {  // Same name
        this.employeeId = employeeId;            // Can be confusing
    }
}
```

### 4. Use Meaningful Variable Names
```java
public class GoodNaming {
    public void process() {
        int customerCount = 100;        // Descriptive
        double totalPrice = 199.99;      // Clear purpose
        boolean isProcessed = false;     // Boolean naming convention
    }
}
```

## Common Scope-Related Errors

### 1. Accessing Out of Scope
```java
public void method() {
    if (true) {
        int x = 10;
    }
    // System.out.println(x);  // Compile error: x not in scope
}
```

### 2. Uninitialized Local Variables
```java
public void method() {
    int x;  // Local variable not initialized
    // System.out.println(x);  // Compile error
}
```

### 3. Shadowing Issues
```java
public class ShadowingIssues {
    private int value = 10;
    
    public void method() {
        int value = 20;  // Shadows instance variable
        System.out.println(value);  // Prints 20, not 10
    }
}
```

Understanding variable scope is essential for writing clean, maintainable Java code and avoiding common programming errors.

Understanding variable scope and lifetime in Java programs

Java Variable Scoping

# System Architecture

System architecture defines the high-level structure of software systems, including components, their relationships, and principles guiding their design and evolution.

## What is System Architecture?

System architecture is the conceptual model that defines the structure, behavior, and views of a system. It serves as the blueprint for both system development and maintenance.

## Architectural Patterns

### Monolithic Architecture
```
┌─────────────────────────────┐
│      Monolithic App         │
├─────────────────────────────┤
│ ┌─────┐ ┌─────┐ ┌─────┐    │
│ │ UI  │ │Logic│ │Data │    │
│ └─────┘ └─────┘ └─────┘    │
└─────────────────────────────┘
```

**Characteristics:**
- Single deployable unit
- Shared database
- Tight coupling
- Simple deployment

**Pros:**
- Easy development and testing
- Simple deployment
- No network latency
- Strong consistency

**Cons:**
- Difficult to scale components
- Technology lock-in
- Single point of failure
- Hard to maintain as it grows

**When to Use:**
- Small applications
- Simple business logic
- Limited team size
- Rapid prototyping

### Microservices Architecture
```
┌──────┐  ┌──────┐  ┌──────┐
│Service│  │Service│  │Service│
│  A    │  │  B    │  │  C    │
└───┬──┘  └───┬──┘  └───┬──┘
    │          │          │
    └──────────┼──────────┘
               │
        ┌──────┴──────┐
        │ API Gateway │
        └──────┬──────┘
               │
        ┌──────┴──────┐
        │    Client   │
        └─────────────┘
```

**Characteristics:**
- Independent services
- Own databases
- Loose coupling
- API communication

**Pros:**
- Independent scaling
- Technology diversity
- Fault isolation
- Team autonomy

**Cons:**
- Network complexity
- Distributed transactions
- Operational overhead
- Testing complexity

**When to Use:**
- Large applications
- Multiple teams
- Different scalability needs
- Technology diversity

### Layered Architecture
```
┌─────────────────┐
│ Presentation    │  ← UI/Controllers
├─────────────────┤
│ Business Logic  │  ← Services/Domain
├─────────────────┤
│ Data Access     │  ← Repositories
├─────────────────┤
│ Database        │  ← Storage
└─────────────────┘
```

**Characteristics:**
- Organized in horizontal layers
- Each layer communicates with adjacent layers
- Separation of concerns

**Benefits:**
- Clear separation of concerns
- Testability
- Maintainability
- Reusability

## Architectural Quality Attributes

### Performance
- **Response Time**: Time to process a request
- **Throughput**: Number of requests per second
- **Latency**: Delay in communication
- **Resource Utilization**: CPU, memory, network usage

### Scalability
```python
# Vertical Scaling (Scale Up)
def scale_vertically():
    # Add more CPU, RAM, storage
    increase_server_resources()

# Horizontal Scaling (Scale Out)
def scale_horizontally():
    # Add more servers
    add_more_servers()
    load_balance_across_servers()
```

### Availability
- **Uptime**: System is operational
- **Fault Tolerance**: System continues despite failures
- **Redundancy**: Backup components

### Security
- **Authentication**: Verify identity
- **Authorization**: Control access
- **Encryption**: Protect data
- **Audit Logging**: Track activities

### Maintainability
- **Modularity**: Independent components
- **Documentation**: Clear architecture docs
- **Testing**: Comprehensive test coverage
- **Code Quality**: Clean, readable code

## Design Principles

### Separation of Concerns
```python
# Bad: Mixed responsibilities
class UserHandler:
    def save_user(self, user):
        # Validation
        if not user.email:
            raise ValueError("Email required")
        
        # Database operations
        db.execute("INSERT INTO users...")
        
        # Email sending
        email_service.send_welcome(user.email)

# Good: Separated concerns
class UserValidator:
    def validate(self, user):
        if not user.email:
            raise ValueError("Email required")

class UserRepository:
    def save(self, user):
        db.execute("INSERT INTO users...")

class EmailService:
    def send_welcome(self, email):
        # Send email logic
```

### Single Responsibility Principle
Each component should have one reason to change.

### Don't Repeat Yourself (DRY)
Avoid code duplication through abstraction.

### Interface Segregation
Clients shouldn't depend on interfaces they don't use.

### Dependency Inversion
Depend on abstractions, not concretions.

## Architectural Decision Making

### Trade-off Analysis
Consider:
- Performance vs. Complexity
- Consistency vs. Availability
- Security vs. Usability
- Cost vs. Features

### Decision Framework
1. **Identify Requirements**: Functional and non-functional
2. **Evaluate Options**: Multiple architectural approaches
3. **Analyze Trade-offs**: Pros and cons of each option
4. **Make Decision**: Choose based on requirements
5. **Document**: Record rationale and assumptions

### Documentation
Document:
- Architectural decisions
- Rationale behind choices
- Assumptions and constraints
- Evolution over time

## Common Architectural Patterns

### Repository Pattern
```python
class UserRepository:
    def __init__(self, db_connection):
        self.db = db_connection
    
    def find_by_id(self, user_id):
        query = "SELECT * FROM users WHERE id = ?"
        return self.db.execute(query, user_id)
    
    def save(self, user):
        query = "INSERT INTO users (name, email) VALUES (?, ?)"
        self.db.execute(query, user.name, user.email)
```

### Factory Pattern
```python
class DatabaseFactory:
    @staticmethod
    def create_database(db_type):
        if db_type == "mysql":
            return MySQLDatabase()
        elif db_type == "postgresql":
            return PostgreSQLDatabase()
        else:
            raise ValueError(f"Unsupported database type: {db_type}")
```

### Observer Pattern
```python
class EventManager:
    def __init__(self):
        self.listeners = {}
    
    def subscribe(self, event_type, listener):
        if event_type not in self.listeners:
            self.listeners[event_type] = []
        self.listeners[event_type].append(listener)
    
    def publish(self, event):
        event_type = event.type
        if event_type in self.listeners:
            for listener in self.listeners[event_type]:
                listener.handle(event)
```

## Architectural Evolution

### Monolith to Microservices
1. **Identify Bounded Contexts**: Natural business boundaries
2. **Extract Services**: Move functionality to services
3. **Implement APIs**: Communication between services
4. **Migrate Data**: Separate databases
5. **Decommission**: Remove old monolith code

### Strangler Fig Pattern
Gradually replace legacy systems with new implementations:
1. Build new functionality alongside existing system
2. Redirect traffic to new system
3. Decommission old components
4. Repeat until fully migrated

## Best Practices

1. **Start Simple**: Begin with monolith, evolve as needed
2. **Measure Everything**: Collect metrics and logs
3. **Design for Failure**: Assume components will fail
4. **Automate Everything**: Deployment, testing, monitoring
5. **Review Regularly**: Architecture should evolve with requirements
6. **Document Decisions**: Record architectural choices and rationale
7. **Consider Team Structure**: Conway's Law - organization influences architecture

## Common Pitfalls

1. **Over-engineering**: Adding unnecessary complexity
2. **Premature Optimization**: Optimizing before measuring
3. **Ignoring Non-functional Requirements**: Focusing only on features
4. **Lack of Monitoring**: No visibility into system behavior
5. **Technology Lock-in**: Difficult to change technologies later

Good system architecture balances competing concerns while providing a foundation for growth and maintenance. It should be simple enough to understand but flexible enough to evolve.

System architecture defines the high-level structure of software systems and their components

System Architecture

# System Scalability

Scalability is the ability of a system to handle a growing amount of load by adding resources. It's a critical aspect of system design that ensures your application can grow with user demand.

## Types of Scaling

### Vertical Scaling (Scale Up)
Increasing the resources of a single machine.

**What to Scale:**
- CPU: More cores, faster processors
- RAM: More memory for larger datasets
- Storage: Faster SSDs, larger capacity
- Network: Higher bandwidth

**Pros:**
- Simple to implement
- No architectural changes needed
- Strong consistency
- Lower complexity

**Cons:**
- Physical limits exist
- Single point of failure
- Expensive at scale
- Downtime during upgrades

**When to Use:**
- Small to medium applications
- Database servers
- Simple workloads
- Rapid scaling needs

### Horizontal Scaling (Scale Out)
Adding more machines to distribute the load.

```
Load Balancer
    │
┌───┼───┐
│   │   │
┌───┐┌───┐┌───┐
│S1 ││S2 ││S3 │
└───┘└───┘└───┘
```

**Pros:**
- Unlimited scaling potential
- Better fault tolerance
- Cost-effective at scale
- Geographic distribution

**Cons:**
- Increased complexity
- Network overhead
- Consistency challenges
- More infrastructure management

**When to Use:**
- Large applications
- High availability requirements
- Global user base
- Variable workloads

## Scalability Patterns

### Load Balancing
Distribute incoming requests across multiple servers.

**Algorithms:**
- **Round Robin**: Sequential distribution
- **Least Connections**: Route to least busy server
- **IP Hash**: Consistent routing based on client IP
- **Weighted Round Robin**: Consider server capacity

```python
class LoadBalancer:
    def __init__(self, servers):
        self.servers = servers
        self.current_index = 0
    
    def round_robin(self):
        server = self.servers[self.current_index]
        self.current_index = (self.current_index + 1) % len(self.servers)
        return server
    
    def least_connections(self):
        return min(self.servers, key=lambda s: s.active_connections)
```

### Caching
Store frequently accessed data in fast storage.

**Levels of Caching:**
1. **Browser Cache**: Client-side caching
2. **CDN Cache**: Edge location caching
3. **Application Cache**: In-memory caching
4. **Database Cache**: Query result caching

**Cache Strategies:**
```python
# Cache-Aside Pattern
def get_user(user_id):
    user = cache.get(f"user:{user_id}")
    if user:
        return user
    
    user = database.get_user(user_id)
    if user:
        cache.set(f"user:{user_id}", user, ttl=3600)
    return user

# Write-Through Pattern
def update_user(user_id, data):
    database.update_user(user_id, data)
    cache.set(f"user:{user_id}", data, ttl=3600)
```

### Database Scaling

#### Read Replicas
```
Primary (Write)    Replica 1 (Read)    Replica 2 (Read)
      │                   │                   │
      └───────────────────┼───────────────────┘
                          │
                   Application Servers
```

**Benefits:**
- Distribute read load
- Improve query performance
- Geographic distribution
- Backup and reporting

**Implementation:**
```sql
-- Primary database
INSERT INTO users (name, email) VALUES ('Alice', 'alice@example.com');

-- Replica databases (read-only)
SELECT * FROM users WHERE email = 'alice@example.com';
```

#### Sharding
Partition data across multiple databases.

**Sharding Strategies:**
- **Horizontal Sharding**: Split rows across tables
- **Vertical Sharding**: Split columns across tables
- **Functional Sharding**: Split by business function

```python
class ShardingManager:
    def __init__(self, shard_count):
        self.shard_count = shard_count
        self.shards = [Database(f"shard_{i}") for i in range(shard_count)]
    
    def get_shard(self, user_id):
        shard_index = user_id % self.shard_count
        return self.shards[shard_index]
    
    def get_user(self, user_id):
        shard = self.get_shard(user_id)
        return shard.get_user(user_id)
```

### Asynchronous Processing
Handle time-consuming tasks asynchronously.

**Message Queues:**
```
Producer → Message Queue → Consumer
```

**Benefits:**
- Decouple components
- Handle traffic spikes
- Improve responsiveness
- Retry mechanisms

```python
# Producer
def send_email(user_id, message):
    queue.publish({
        'type': 'email',
        'user_id': user_id,
        'message': message
    })

# Consumer
def process_email_queue():
    while True:
        message = queue.consume()
        if message:
            email_service.send(
                message['user_id'],
                message['message']
            )
```

## Performance Optimization

### Database Optimization

#### Indexing
```sql
-- Create index for frequent queries
CREATE INDEX idx_users_email ON users(email);

-- Composite index for multiple columns
CREATE INDEX idx_orders_user_date ON orders(user_id, order_date);
```

#### Query Optimization
```sql
-- Bad: Full table scan
SELECT * FROM users WHERE email LIKE '%@gmail.com';

-- Good: Index-friendly query
SELECT * FROM users WHERE email = 'alice@gmail.com';

-- Use EXPLAIN to analyze queries
EXPLAIN SELECT * FROM users WHERE email = 'alice@gmail.com';
```

### Application Optimization

#### Connection Pooling
```python
class ConnectionPool:
    def __init__(self, max_connections):
        self.max_connections = max_connections
        self.available_connections = []
        self.used_connections = set()
    
    def get_connection(self):
        if self.available_connections:
            conn = self.available_connections.pop()
        elif len(self.used_connections) < self.max_connections:
            conn = self.create_connection()
        else:
            raise Exception("No available connections")
        
        self.used_connections.add(conn)
        return conn
    
    def release_connection(self, conn):
        self.used_connections.remove(conn)
        self.available_connections.append(conn)
```

#### Batching Operations
```python
# Bad: Individual database calls
for user in users:
    database.save_user(user)

# Good: Batch operations
database.save_users_batch(users)
```

## Monitoring Scalability

### Key Metrics
- **Throughput**: Requests per second
- **Response Time**: Average and percentiles
- **Error Rate**: Failed requests percentage
- **Resource Utilization**: CPU, memory, disk, network
- **Queue Depth**: Pending requests

### Monitoring Tools
- **Application Performance Monitoring (APM)**: New Relic, DataDog
- **Infrastructure Monitoring**: Prometheus, Grafana
- **Log Analysis**: ELK Stack, Splunk
- **Distributed Tracing**: Jaeger, Zipkin

## Auto Scaling

### Cloud Auto Scaling
```python
# AWS Auto Scaling example
auto_scaling_group = AutoScalingGroup(
    min_size=2,
    max_size=10,
    desired_capacity=4,
    launch_configuration=launch_config,
    scaling_policies=[
        ScalingPolicy(
            name='scale_up',
            adjustment_type='ChangeInCapacity',
            scaling_adjustment=2,
            cooldown=300
        )
    ]
)
```

### Custom Metrics
- CPU utilization > 70%
- Memory usage > 80%
- Queue length > 1000
- Response time > 500ms

## Best Practices

1. **Design for Scale from Start**: Architecture should support future growth
2. **Measure Before Optimizing**: Use data to identify bottlenecks
3. **Use Caching Strategically**: Cache at multiple levels
4. **Implement Circuit Breakers**: Prevent cascading failures
5. **Design for Failure**: Assume components will fail
6. **Monitor Everything**: Comprehensive observability
7. **Test at Scale**: Load testing with realistic traffic

## Common Scalability Challenges

### Hot Spotting
Uneven distribution of load across resources.

**Solutions:**
- Consistent hashing
- Better sharding keys
- Load-aware routing

### Database Bottlenecks
Database becomes the limiting factor.

**Solutions:**
- Read replicas
- Sharding
- Caching layers
- NoSQL alternatives

### Network Latency
Communication overhead between components.

**Solutions:**
- Geographic distribution
- CDNs
- Connection pooling
- Data compression

### State Management
Managing state across distributed systems.

**Solutions:**
- External session stores
- Eventual consistency
- Distributed caches
- Stateless services

## Scalability vs. Consistency

### CAP Theorem
You can only have two of three:
- **Consistency**: All nodes see same data
- **Availability**: System remains operational
- **Partition Tolerance**: Handle network failures

### Trade-offs
```python
# Strong Consistency (CP)
def transfer_money(from_account, to_account, amount):
    with database.transaction():
        from_account.withdraw(amount)
        to_account.deposit(amount)

# Eventual Consistency (AP)
def update_profile(user_id, profile_data):
    database.update_user(user_id, profile_data)
    search_index.update_user(user_id, profile_data)
    cache.invalidate_user(user_id)
    # Updates propagate asynchronously
```

Scalability is not just about handling more users—it's about maintaining performance, reliability, and cost-effectiveness as your system grows.

System scalability is the ability of a system to handle a growing amount of load by adding resources

System Scalability

# Distributed Systems

Distributed systems are collections of independent computers that appear to users as a single coherent system. They enable building scalable, fault-tolerant, and globally accessible applications.

## What is a Distributed System?

A distributed system is a system whose components are located on different networked computers, which communicate and coordinate their actions by passing messages to one another.

## Characteristics

### Concurrency
Multiple components execute simultaneously.

### Lack of Global Clock
No single, globally agreed-upon time reference.

### Independent Failures
Components can fail independently without affecting the entire system.

## Challenges

### Network Communication
- **Latency**: Delays in message delivery
- **Bandwidth**: Limited data transfer capacity
- **Reliability**: Messages can be lost, duplicated, or reordered

### Consistency
Ensuring all nodes have the same view of data.

### Fault Tolerance
System continues operating despite component failures.

### Coordination
Managing interactions between distributed components.

## Architectural Patterns

### Client-Server Architecture
```
┌──────┐    ┌──────┐    ┌──────┐
│Client│◄───│Server│◄───│Client│
└──────┘    └──────┘    └──────┘
```

**Characteristics:**
- Centralized server
- Multiple clients
- Request-response model

**Use Cases:**
- Web applications
- Database systems
- File servers

### Peer-to-Peer Architecture
```
┌─────┐    ┌─────┐    ┌─────┐
│Peer │◄───│Peer │◄───│Peer │
│  A  │    │  B  │    │  C  │
└─────┘    └─────┘    └─────┘
```

**Characteristics:**
- No central server
- All nodes are equal
- Direct communication between peers

**Use Cases:**
- File sharing (BitTorrent)
- Blockchain networks
- Distributed databases

### Microservices Architecture
```
┌──────┐  ┌──────┐  ┌──────┐
│Service│  │Service│  │Service│
│  A    │  │  B    │  │  C    │
└───┬──┘  └───┬──┘  └───┬──┘
    │          │          │
    └──────────┼──────────┘
               │
        ┌──────┴──────┐
        │ API Gateway │
        └──────┬──────┘
               │
        ┌──────┴──────┐
        │    Client   │
        └─────────────┘
```

**Characteristics:**
- Independent services
- Own databases
- API communication
- Loose coupling

## Consistency Models

### Strong Consistency
All nodes see the same data at the same time.

```python
class StrongConsistencyStore:
    def write(self, key, value):
        # Synchronous replication to all nodes
        for node in self.nodes:
            node.write(key, value)
        
        # Wait for acknowledgment from all nodes
        self.wait_for_acks()
    
    def read(self, key):
        # Read from any node (all have same data)
        return self.nodes[0].read(key)
```

### Eventual Consistency
System will become consistent over time, but may have temporary inconsistencies.

```python
class EventualConsistencyStore:
    def write(self, key, value):
        # Write to primary node
        self.primary.write(key, value)
        
        # Asynchronously replicate to other nodes
        for replica in self.replicas:
            self.async_replicate(replica, key, value)
    
    def read(self, key):
        # Read from nearest node
        return self.get_nearest_node().read(key)
```

### Causal Consistency
Operations that are causally related are seen in the same order by all nodes.

```python
class CausalConsistencyStore:
    def write(self, key, value, dependencies):
        # Include causal dependencies
        operation = {
            'key': key,
            'value': value,
            'dependencies': dependencies,
            'timestamp': self.get_logical_clock()
        }
        
        self.broadcast_operation(operation)
```

## Consensus Algorithms

### Paxos
Algorithm for achieving consensus in distributed systems.

**Roles:**
- **Proposer**: Proposes values
- **Acceptor**: Votes on proposals
- **Learner**: Learns the decided value

**Phases:**
1. **Prepare**: Proposer asks acceptors to promise not to accept older proposals
2. **Accept**: Proposer sends value to acceptors
3. **Learn**: Acceptors notify learners of accepted value

### Raft
Simpler consensus algorithm for practical systems.

**States:**
- **Follower**: Passive state, responds to leaders
- **Candidate**: Campaigning to become leader
- **Leader**: Handles all client requests

**Election Process:**
```python
class RaftNode:
    def __init__(self):
        self.state = 'follower'
        self.term = 0
        self.voted_for = None
        self.log = []
    
    def start_election(self):
        self.state = 'candidate'
        self.term += 1
        self.voted_for = self.id
        
        votes = 1  # Vote for self
        for peer in self.peers:
            if peer.request_vote(self.term, self.id):
                votes += 1
        
        if votes > len(self.peers) / 2:
            self.become_leader()
```

## Distributed Databases

### Sharding
Partition data across multiple nodes.

**Sharding Strategies:**
- **Range-based**: Partition by key ranges
- **Hash-based**: Partition by hash of key
- **Directory-based**: Central directory maps keys to nodes

```python
class ShardedDatabase:
    def __init__(self, shards):
        self.shards = shards
        self.hash_function = consistent_hash
    
    def get_shard(self, key):
        return self.shards[self.hash_function(key)]
    
    def get(self, key):
        shard = self.get_shard(key)
        return shard.get(key)
    
    def put(self, key, value):
        shard = self.get_shard(key)
        shard.put(key, value)
```

### Replication
Maintain copies of data on multiple nodes.

**Replication Strategies:**
- **Master-Slave**: One master, multiple slaves
- **Multi-Master**: Multiple masters
- **Leaderless**: No single master

```python
class ReplicatedDatabase:
    def __init__(self, nodes, replication_factor=3):
        self.nodes = nodes
        self.replication_factor = replication_factor
    
    def write(self, key, value):
        # Write to multiple nodes
        target_nodes = self.get_replication_nodes(key)
        
        for node in target_nodes:
            node.write(key, value)
        
        # Wait for quorum
        self.wait_for_quorum(target_nodes)
    
    def read(self, key):
        # Read from multiple nodes for consistency
        target_nodes = self.get_replication_nodes(key)
        responses = []
        
        for node in target_nodes:
            response = node.read(key)
            responses.append(response)
        
        # Resolve conflicts if any
        return self.resolve_conflicts(responses)
```

## Message Passing

### Remote Procedure Calls (RPC)
Call procedures on remote machines as if they were local.

```python
class RPCClient:
    def __init__(self, server_address):
        self.server_address = server_address
    
    def call(self, method, *args):
        request = {
            'method': method,
            'args': args,
            'id': self.generate_id()
        }
        
        response = self.send_request(request)
        return response['result']

# Usage
client = RPCClient('server:8080')
result = client.call('add', 5, 3)  # Returns 8
```

### Message Queues
Asynchronous communication between components.

```python
class MessageQueue:
    def __init__(self):
        self.queue = []
        self.subscribers = {}
    
    def publish(self, topic, message):
        for subscriber in self.subscribers.get(topic, []):
            subscriber.receive(message)
    
    def subscribe(self, topic, callback):
        if topic not in self.subscribers:
            self.subscribers[topic] = []
        self.subscribers[topic].append(callback)

# Usage
mq = MessageQueue()
mq.subscribe('orders', process_order)
mq.publish('orders', {'id': 123, 'amount': 100})
```

## Fault Tolerance

### Redundancy
Multiple copies of components to handle failures.

### Failover
Automatic switching to backup components.

```python
class FailoverService:
    def __init__(self, primary, backup):
        self.primary = primary
        self.backup = backup
        self.current = primary
    
    def call(self, method, *args):
        try:
            return self.current.call(method, *args)
        except Exception as e:
            if self.current == self.primary:
                self.current = self.backup
                return self.current.call(method, *args)
            else:
                raise e
```

### Circuit Breaker
Prevent cascading failures.

```python
class CircuitBreaker:
    def __init__(self, failure_threshold=5, timeout=60):
        self.failure_threshold = failure_threshold
        self.timeout = timeout
        self.failure_count = 0
        self.last_failure_time = None
        self.state = 'CLOSED'
    
    def call(self, func, *args, **kwargs):
        if self.state == 'OPEN':
            if self._should_attempt_reset():
                self.state = 'HALF_OPEN'
            else:
                raise Exception("Circuit breaker is OPEN")
        
        try:
            result = func(*args, **kwargs)
            self._on_success()
            return result
        except Exception as e:
            self._on_failure()
            raise e
```

## Distributed Caching

### Consistent Hashing
Distribute cache keys evenly across nodes.

```python
class ConsistentHashRing:
    def __init__(self, nodes, replicas=100):
        self.ring = {}
        self.sorted_keys = []
        
        for node in nodes:
            for i in range(replicas):
                key = self.hash(f"{node}:{i}")
                self.ring[key] = node
                self.sorted_keys.append(key)
        
        self.sorted_keys.sort()
    
    def get_node(self, key):
        if not self.ring:
            return None
        
        hash_key = self.hash(key)
        idx = bisect.bisect_right(self.sorted_keys, hash_key)
        
        if idx == len(self.sorted_keys):
            idx = 0
        
        return self.ring[self.sorted_keys[idx]]
```

## Monitoring Distributed Systems

### Distributed Tracing
Track requests as they flow through multiple services.

```python
class DistributedTracer:
    def __init__(self):
        self.spans = []
    
    def start_span(self, operation_name, parent_span=None):
        span = {
            'trace_id': self.generate_trace_id(),
            'span_id': self.generate_span_id(),
            'parent_span_id': parent_span['span_id'] if parent_span else None,
            'operation_name': operation_name,
            'start_time': time.time()
        }
        
        self.spans.append(span)
        return span
    
    def finish_span(self, span):
        span['end_time'] = time.time()
        span['duration'] = span['end_time'] - span['start_time']
```

### Health Checks
Monitor the health of distributed components.

```python
class HealthChecker:
    def __init__(self, services):
        self.services = services
    
    def check_all(self):
        results = {}
        for service_name, service in self.services.items():
            try:
                health = service.health_check()
                results[service_name] = {
                    'status': 'healthy',
                    'details': health
                }
            except Exception as e:
                results[service_name] = {
                    'status': 'unhealthy',
                    'error': str(e)
                }
        return results
```

## Best Practices

1. **Design for Failure**: Assume components will fail
2. **Use Idempotent Operations**: Safe retry mechanisms
3. **Implement Timeouts**: Prevent hanging operations
4. **Monitor Everything**: Comprehensive observability
5. **Use Circuit Breakers**: Prevent cascading failures
6. **Implement Retry Logic**: Handle transient failures
7. **Design for Scalability**: Architecture should support growth

Distributed systems enable building powerful, scalable applications but require careful design to handle the complexities of network communication, consistency, and fault tolerance.

Distributed systems are collections of independent computers that appear as a single coherent system

Distributed Systems

# System Design

System design involves designing and architecting complex software systems that are scalable, reliable, and maintainable.

## What is System Design?

System design is the process of defining the architecture, components, modules, interfaces, and data for a system to satisfy specified requirements.

## Key Concepts

### Scalability
Ability of a system to handle growing amounts of load.

**Types:**
- **Vertical Scaling**: Increasing resources of a single machine
- **Horizontal Scaling**: Adding more machines to distribute load

### Reliability
Probability that a system will function correctly over time.

**Techniques:**
- Redundancy
- Failover mechanisms
- Load balancing
- Circuit breakers

### Availability
Percentage of time the system is operational.

**High Availability (HA):**
- 99.9% = 8.76 hours downtime/year
- 99.99% = 52.56 minutes downtime/year
- 99.999% = 5.26 minutes downtime/year

### Performance
System's responsiveness and throughput under various conditions.

**Metrics:**
- **Latency**: Time to process a request
- **Throughput**: Requests per second
- **Response Time**: Total time for request completion

## System Design Process

### 1. Requirements Gathering
Understand functional and non-functional requirements.

**Functional Requirements:**
- What the system should do
- User stories and use cases
- Business logic

**Non-Functional Requirements:**
- Performance targets
- Scalability needs
- Security requirements
- Availability goals

### 2. Estimation
Calculate storage, bandwidth, and traffic requirements.

**Storage Estimation:**
```python
# Example: Photo sharing service
users_per_day = 1000
photos_per_user = 3
photo_size_mb = 2
retention_days = 365

daily_storage = users_per_day * photos_per_user * photo_size_mb
yearly_storage = daily_storage * retention_days
```

**Traffic Estimation:**
- Read/write ratio
- Peak traffic patterns
- Growth projections

### 3. High-Level Design
Create overall system architecture.

**Components:**
- Load balancers
- Web servers
- Application servers
- Databases
- Caching layers
- Message queues

### 4. Detailed Design
Design individual components and their interactions.

**Consider:**
- Data models
- API design
- Database schema
- Caching strategy
- Security measures

### 5. Identify Bottlenecks
Find potential performance limitations.

**Common Bottlenecks:**
- Database queries
- Network latency
- Memory usage
- I/O operations

## Common Architectural Patterns

### Microservices Architecture
```
┌──────┐  ┌──────┐  ┌──────┐
│Service│  │Service│  │Service│
│  A    │  │  B    │  │  C    │
└───┬──┘  └───┬──┘  └───┬──┘
    │          │          │
    └──────────┼──────────┘
               │
        ┌──────┴──────┐
        │ API Gateway │
        └──────┬──────┘
               │
        ┌──────┴──────┐
        │    Client   │
        └─────────────┘
```

**Advantages:**
- Independent scaling
- Technology diversity
- Fault isolation
- Team autonomy

**Challenges:**
- Network complexity
- Distributed transactions
- Service discovery
- Monitoring complexity

### Event-Driven Architecture
```
┌──────┐    ┌──────┐    ┌──────┐
│Event │    │Event │    │Event │
│Source│───►│ Bus  │───►│Handler│
└──────┘    └──────┘    └──────┘
```

**Components:**
- Event producers
- Event bus/broker
- Event consumers
- Event stores

### Serverless Architecture
```
┌──────────┐    ┌──────────┐    ┌──────────┐
│ Function │    │ Function │    │ Function │
│ Service  │    │ Service  │    │ Service  │
└────┬─────┘    └────┬─────┘    └────┬─────┘
     │                │                │
     └────────────────┼────────────────┘
                      │
              ┌───────┴───────┐
              │   Cloud       │
              │   Provider    │
              └───────────────┘
```

**Benefits:**
- No server management
- Automatic scaling
- Pay-per-use
- Reduced operational complexity

## Database Design

### SQL vs NoSQL

| Factor | SQL | NoSQL |
|--------|-----|-------|
| Schema | Fixed | Flexible |
| Scaling | Vertical | Horizontal |
| Consistency | Strong | Eventual |
| Query Language | SQL | Various |
| Use Case | Structured data | Unstructured data |

### Database Patterns

**Read Replicas:**
- Primary database for writes
- Multiple replicas for reads
- Improves read performance

**Sharding:**
- Partition data across multiple databases
- Horizontal scaling
- Complex to implement

**CQRS:**
- Command Query Responsibility Segregation
- Separate models for read and write
- Optimized for different access patterns

## Caching Strategies

### Cache-Aside Pattern
```python
def get_user(user_id):
    # Check cache first
    user = cache.get(f"user:{user_id}")
    if user:
        return user
    
    # Load from database
    user = db.get_user(user_id)
    if user:
        cache.set(f"user:{user_id}", user, ttl=3600)
    
    return user
```

### Write-Through Cache
- Write to cache and database simultaneously
- Ensures cache consistency
- Higher write latency

### Write-Behind Cache
- Write to cache immediately
- Asynchronously write to database
- Better write performance
- Risk of data loss

## Load Balancing

### Algorithms
- **Round Robin**: Sequential distribution
- **Least Connections**: Route to least busy server
- **IP Hash**: Consistent routing based on IP
- **Weighted Round Robin**: Consider server capacity

### Types
- **Layer 4**: Transport layer (TCP/UDP)
- **Layer 7**: Application layer (HTTP)

## Security Considerations

### Authentication & Authorization
- OAuth 2.0
- JWT tokens
- Role-based access control (RBAC)
- Multi-factor authentication

### Data Protection
- Encryption at rest
- Encryption in transit
- Data masking
- Access logging

### Network Security
- Firewalls
- DDoS protection
- Rate limiting
- SSL/TLS certificates

## Monitoring and Observability

### The Three Pillars
1. **Logs**: Event records
2. **Metrics**: Numerical measurements
3. **Traces**: Request flows

### Key Metrics
- **SLA**: Service Level Agreement compliance
- **SLO**: Service Level Objective
- **SLI**: Service Level Indicator
- **Error Rate**: Percentage of failed requests
- **Latency**: Response time percentiles

## Design Examples

### URL Shortener
**Requirements:**
- Generate short URLs
- Redirect to original URLs
- Handle high traffic
- Analytics and tracking

**Design:**
- Hash function for URL generation
- Distributed cache for hot URLs
- Database for persistence
- CDN for global performance

### Messaging System
**Requirements:**
- Real-time messaging
- Group chats
- Message history
- Online status

**Design:**
- WebSocket connections
- Message queues
- NoSQL for messages
- Redis for online status

## Best Practices

1. **Start Simple**: Begin with monolith, evolve as needed
2. **Design for Failure**: Assume components will fail
3. **Measure Everything**: Collect comprehensive metrics
4. **Automate Everything**: Deployment, testing, monitoring
5. **Security First**: Design with security in mind
6. **Document Decisions**: Record architectural choices
7. **Review Regularly**: Architecture should evolve

System design is both an art and science, requiring balance between competing requirements and constraints.

Designing and architecting complex software systems that are scalable, reliable, and maintainable

System Design

# 100xSystems Learning Paths

This repository contains structured learning content organized by paths and topics for comprehensive learning.

## Available Paths

- **Foundations**: Programming fundamentals including Java, Python, JavaScript, and Data Structures
- **System Design**: Architecture, scalability, and distributed systems
- **Development**: Tools and methodologies (coming soon)
- **Design Patterns**: OOP and architectural patterns (coming soon)
- **Security**: Security principles and cryptography (coming soon)
- **Optimization**: Performance tuning and scalability (coming soon)

Structured learning content organized by paths and topics for comprehensive learning

100xSystems Learning Paths

Modules

1Foundations

2System Design

About This Path

100xSystems Learning Paths

Available Paths

Platform

System Architecture

Design Patterns

Development

DevOps & Tools

Resources

Community

Connect

Legal