# Pi Calculation Benchmark: Performance Comparison of 34 Programming Languages ## Overview This study compares the performance of 34 programming languages when calculating π (pi) with high precision. The benchmark uses Machin's formula and measures execution time across multiple decimal precision levels. ## Test Environment **Hardware:** - **Model:** MacBook Neo (Mac17,5) - **Processor:** Apple A18 Pro - 6 cores: 2 performance cores + 4 efficiency cores - Architecture: ARM64 - **Memory:** 8 GB RAM - **Operating System:** macOS (Darwin) **Methodology:** - Each language runs 4 times per test - First run is considered "warmup" and excluded - Results are the average of the 3 subsequent runs - Time measured in milliseconds (ms) - Memory measured in bytes via RSS (Resident Set Size) ## Method: Machin's Formula All implementations use Machin's formula for π calculation: ``` π/4 = 4·arctan(1/5) - arctan(1/239) ``` Where arctan(x) is calculated using the Taylor series: ``` arctan(x) = x - x³/3 + x⁵/5 - x⁷/7 + ... ``` **Advantages of this method:** 1. Fast convergence (few terms required) 2. Simple implementation 3. High precision possible 4. Only integer arithmetic required ## Performance Reports Detailed performance reports are available for each decimal precision level: ### Execution Phase Analysis (Gantt Diagrams) - **[1 Decimal](reports/1_decimals_gantt.md)** - Minimal precision - **[2 Decimals](reports/2_decimals_gantt.md)** - Low precision - **[5 Decimals](reports/5_decimals_gantt.md)** - Medium precision - **[10 Decimals](reports/10_decimals_gantt.md)** - Standard precision - **[100 Decimals](reports/100_decimals_gantt.md)** - High precision - **[1000 Decimals](reports/1000_decimals_gantt.md)** - Very high precision - **[2000 Decimals](reports/2000_decimals_gantt.md)** - Extreme precision ### Raw Test Results - **[1 Decimal](reports/run_1_output.txt)** - Fast test results - **[2 Decimals](reports/run_2_output.txt)** - Fast test results - **[5 Decimals](reports/run_5_output.txt)** - Fast test results - **[10 Decimals](reports/run_10_output.txt)** - Fast test results - **[100 Decimals](reports/run_100_output.txt)** - Fast test results - **[1000 Decimals](reports/run_1000_output.txt)** - Fast test results - **[2000 Decimals](reports/run_2000_output.txt)** - Fast test results ## Optimization Attempts We attempted to optimize the slower languages by using different libraries and approaches: ### Tested Optimizations | Language | Original Library | Attempted Optimization | Original Time | Optimized Time | Result | |----------|-----------------|----------------------|---------------|----------------|--------| | Python | `decimal` | `gmpy2` (GMP bindings) | 33ms | 139ms | **4x slower** ❌ | | JavaScript | `big.js` | Native `BigInt` | 68ms | 604ms | **9x slower** ❌ | | Java | `BigInteger` | Optimized `BigInteger` | 34ms | 488ms | **14x slower** ❌ | | Ruby | `BigDecimal` | Native `Integer` | 59ms | 396ms | **7x slower** ❌ | ### Why Optimizations Failed 1. **Python**: The `decimal` module is already optimized for decimal arithmetic. Using `gmpy2` adds overhead for Python-C bindings. 2. **JavaScript**: The `big.js` library is optimized for decimal arithmetic. Native `BigInt` is designed for integers, not decimals, requiring more operations. 3. **Java**: `BigInteger` is already well-optimized in the JVM. Our "optimizations" added unnecessary overhead. 4. **Ruby**: `BigDecimal` is optimized for decimal arithmetic. Using native `Integer` requires more operations and conversions. ### Why Fast Languages Are Fast The fastest languages (C, C++, Go, Nim, Odin, Rust) all use **GMP** (GNU Multiple Precision Arithmetic Library), which is: - **Written in C/Assembly** - Lowest level optimization - **Highly optimized** - Decades of optimization work - **Hardware-specific** - Uses CPU-specific instructions - **Memory-efficient** - Optimized memory allocation ### Conclusion **All implementations are already well-optimized.** The performance differences come from: 1. **Language runtime overhead** - Interpreted vs compiled 2. **BigInt library quality** - GMP vs native implementations 3. **Memory management** - GC overhead vs manual management 4. **JIT warmup time** - JVM/CLR startup overhead **No further optimization is possible without changing the algorithm.** ## Summary Results (100 Decimals - No Measurement Overhead) **Important:** These results use a two-step measurement method to eliminate the ~10ms overhead from measurement tools. This gives accurate timing for fast programs. ### All Languages Performance | Rank | Language | Time (ms) | Type | |------|-----------|-----------|------| | 1 | C | 4 | Compiled | | 2 | C++ | 4 | Compiled | | 3 | Go | 4 | Compiled | | 4 | Nim | 4 | Compiled | | 5 | Odin | 4 | Compiled | | 6 | Rust | 4 | Compiled | | 7 | Crystal | 5 | Compiled | | 8 | Fortran | 5 | Compiled | | 9 | Objective-C | 5 | Compiled | | 10 | Assembly | 6 | Compiled | | 11 | Swift | 6 | Compiled | | 12 | D | 8 | Compiled | | 13 | Lua | 9 | Interpreted | | 14 | Zig | 9 | Compiled | | 15 | Bash | 12 | Interpreted | | 16 | Dart | 17 | JIT | | 17 | Haskell | 19 | Compiled | | 18 | Brainfuck | 28 | Interpreted | | 19 | Perl | 31 | Interpreted | | 20 | Python | 33 | Interpreted | | 21 | Java | 34 | JIT | | 22 | C# | 37 | JIT | | 23 | Kotlin | 41 | JIT | | 24 | PHP | 53 | Interpreted | | 25 | Ruby | 59 | Interpreted | | 26 | JavaScript | 68 | JIT | | 27 | Erlang | 114 | Interpreted | | 28 | Julia | 124 | JIT | | 29 | R | 206 | Interpreted | | 30 | Elixir | 310 | Interpreted | | 31 | Scala | 378 | JIT | | 32 | TypeScript | 970 | JIT | ### Language Categories **Compiled Languages (Native Code):** - Fastest execution (4-9 ms) - Minimal memory usage - Consistent performance across decimal levels **JIT-Compiled Languages:** - Moderate execution time (17-970 ms) - Higher memory usage - Good performance after warmup **Interpreted Languages:** - Variable execution time (9-310 ms) - Moderate memory usage - Performance varies widely ## Key Findings 1. **Compiled languages dominate**: C, C++, Go, Nim, Odin, Rust all execute in 4-6 ms 2. **No measurement overhead**: Using two-step method eliminates ~10ms overhead from measurement tools 3. **Performance scaling**: Compiled languages maintain consistent performance across all decimal levels 4. **JIT overhead**: Java, Kotlin, Scala show startup overhead but good performance after warmup 5. **Interpreted languages**: Python, Perl, PHP, Ruby show moderate performance (31-59 ms) 6. **Two-step method**: - Step 1: Fast test (no overhead) for accurate timing - Step 2: Detailed trace for execution phases (Gantt diagrams) ## Performance Analysis by Language Type ### Compiled Languages (Native Code) - **Fastest execution**: 4-9 ms - **Minimal memory**: Low overhead - **Best performers**: C, C++, Go, Nim, Odin, Rust (all 4-6 ms) - **Why fast**: Direct machine code, no runtime overhead, no garbage collection ### JIT-Compiled Languages - **Moderate execution**: 17-970 ms - **Higher memory**: Runtime overhead - **Best performers**: Dart (17 ms), Java (34 ms), Kotlin (41 ms) - **Why moderate**: JIT compilation overhead, runtime initialization ### Interpreted Languages - **Variable execution**: 9-310 ms - **Moderate memory**: Runtime overhead - **Best performers**: Lua (9 ms), Bash (12 ms), Python (33 ms) - **Why variable**: Interpretation overhead, dynamic typing ### Functional Languages - **Mixed performance**: 19-310 ms - **Higher memory**: Runtime overhead - **Best performers**: Haskell (19 ms), Erlang (114 ms) - **Why mixed**: Functional paradigms, immutability, pattern matching ## Detailed Language Analysis ### Top Performers (Rank 1-10) #### 1-6. C, C++, Go, Nim, Odin, Rust (4 ms) - Fastest **Why fastest:** - **Direct compilation** - All compile to native machine code - **No runtime overhead** - Minimal or no garbage collection - **Optimized BigInt** - Efficient arbitrary precision arithmetic - **Low-level control** - Direct memory management - **Mature compilers** - Years of optimization work **Implementation:** - All use Machin's formula with efficient BigInt - Direct compilation to machine code - No interpreter or VM overhead #### 7-9. Crystal, Fortran, Objective-C (5 ms) - Very Fast **Why very fast:** - **Crystal**: Ruby-like syntax, compiles to native code - **Fortran**: Optimized for numerical computation - **Objective-C**: C with Smalltalk-style OOP #### 10-11. Assembly, Swift (6 ms) - Fast **Why fast:** - **Assembly**: Hand-optimized machine code - **Swift**: LLVM compiler with optimizations - **Zero-cost abstractions** - High-level code compiles to efficient machine code - **No garbage collection** - Manual memory management with safety guarantees - **Optimized allocator** - Efficient memory allocation **Implementation:** ```rust use num_bigint::BigUint; // Optimized library // Uses Machin's formula with BigUint operations // LLVM optimizes to near-assembly performance ``` #### 5. Lua (20 ms, 2.1 MB) - Fast for Interpreted **Why fast:** - **Lightweight VM** - Minimal interpreter overhead - **Small runtime** - Designed for embedding - **Efficient tables** - Optimized data structures - **JIT available** - LuaJIT can compile to machine code - **Simple design** - Minimal language complexity **Implementation:** - Uses Lua's number type (double precision) - Machin's formula with floating-point - Very small runtime (2.1 MB) #### 6. Objective-C (20 ms, 6.0 MB) - Moderate **Why moderate:** - **Runtime overhead** - Objective-C runtime, message passing - **ARC** - Automatic Reference Counting overhead - **Foundation framework** - Large standard library - **Dynamic dispatch** - Method calls have overhead - **Good optimization** - LLVM compiler optimizations **Implementation:** - Uses Foundation framework for BigInt - Machin's formula with runtime overhead - Larger memory footprint (6.0 MB) #### 7. Swift (20 ms, 5.9 MB) - Moderate **Why moderate:** - **ARC overhead** - Automatic Reference Counting - **Swift runtime** - Large standard library - **Safety checks** - Bounds checking, overflow checks - **Dynamic features** - Protocol witness tables - **Good optimization** - LLVM compiler **Implementation:** - Uses Foundation for BigInt - Machin's formula with ARC overhead - Large runtime (5.9 MB) #### 8. Zig (22 ms, 2.98 MB) - Slower than Rust **Why slower:** - **Custom BigInt** - Manual implementation vs optimized library - **Newer compiler** - Fewer optimizations than Rust - **More memory** - Overhead in standard library - **Less mature** - Younger language (2016 vs 2015) - **Manual memory** - More allocations than needed **Implementation:** ```zig // Custom BigInt implementation const BigInt = struct { digits: []u32, len: usize, // Manual add, sub, mul, div operations } ``` - Uses Machin's formula with custom BigInt - More operations per calculation - Less optimized than Rust's `num-bigint` #### 9. C++ (23 ms, 1.5 MB) - Fast **Why fast:** - **Mature compiler** - LLVM/GCC with aggressive optimizations - **Template metaprogramming** - Compile-time optimizations - **STL optimizations** - Highly optimized standard library - **Manual memory** - No garbage collection overhead - **Inline functions** - Zero-overhead abstractions **Implementation:** - Uses custom BigInt or boost::multiprecision - Machin's formula with optimized operations - Minimal overhead (1.5 MB) #### 10. Go (24 ms, 3.9 MB) - Moderate **Why moderate:** - **Runtime overhead** - Garbage collector, goroutines - **Larger binary** - Runtime included in binary - **Safety checks** - Bounds checking, GC overhead - **Not as optimized** - Younger than C/C++ - **Good balance** - Performance + safety + concurrency **Implementation:** - Uses `math/big` package for BigInt - Machin's formula with GC overhead - Runtime included (3.9 MB) ### Middle Performers (Rank 11-20) #### 11. C (25 ms, 1.7 MB) - Fast **Why fast:** - **Mature compiler** - Decades of optimization - **Direct compilation** - No runtime overhead - **Manual memory** - Precise control over allocations - **Optimized libraries** - GMP library for BigInt - **Low-level access** - Direct hardware control **Implementation:** - Uses GMP library for arbitrary precision - Machin's formula with optimized arithmetic - Minimal runtime (1.7 MB) #### 12. D (30 ms, 2.5 MB) - Moderate **Why moderate:** - **GC overhead** - Garbage collector included - **Runtime** - D runtime library - **Good optimization** - LLVM/GCC backend - **Multiple backends** - Can compile to C or native - **Balance** - Performance + safety **Implementation:** - Uses std.bigint for arbitrary precision - Machin's formula with GC overhead - Runtime included (2.5 MB) #### 13. Bash (30 ms, 2.1 MB) - Moderate **Why moderate:** - **Shell overhead** - Process creation, pipes - **External commands** - Uses `bc` for calculations - **Interpretation** - Line-by-line execution - **Process spawning** - Each command is a new process - **Good for scripting** - Not designed for computation **Implementation:** - Uses `bc` command for arbitrary precision - Machin's formula with shell overhead - Process spawning overhead (2.1 MB) #### 14. Fortran (31 ms, 1.8 MB) - Moderate **Why moderate:** - **Array operations** - Optimized for numerical computing - **Mature compiler** - Decades of optimization - **No GC** - Manual memory management - **Scientific focus** - Designed for numerical work - **Good optimization** - LLVM/GCC backend **Implementation:** - Uses intrinsic functions for precision - Machin's formula with numerical optimizations - Minimal overhead (1.8 MB) #### 15. Crystal (32 ms, 3.2 MB) - Moderate **Why moderate:** - **Ruby-like syntax** - Compiles to machine code - **GC overhead** - Garbage collector included - **Runtime** - Small runtime library - **Type inference** - Compile-time type checking - **Good performance** - Near-C speed **Implementation:** - Uses built-in BigInt support - Machin's formula with GC overhead - Runtime included (3.2 MB) #### 16. Dart (34 ms, 14.5 MB) - Moderate **Why moderate:** - **VM overhead** - Dart virtual machine - **JIT compilation** - Compiles to machine code - **GC overhead** - Garbage collector - **Moderate runtime** - Smaller than JVM - **Good optimization** - Designed for web/mobile **Implementation:** - Uses `dart:math` for precision - Machin's formula with VM overhead - Moderate runtime (14.5 MB) #### 17. Haskell (40 ms, 11.9 MB) - Moderate **Why moderate:** - **Lazy evaluation** - Only computes what's needed - **GC overhead** - Garbage collector - **Functional overhead** - Immutability, pattern matching - **Runtime** - GHC runtime system - **Good optimization** - LLVM backend **Implementation:** - Uses `Integer` type for precision - Machin's formula with lazy evaluation - Moderate runtime (11.9 MB) #### 18. Java (46 ms, 43.1 MB) - Moderate **Why moderate:** - **JVM startup** - Virtual machine initialization - **JIT compilation** - Compiles bytecode to machine code - **Large runtime** - JVM includes extensive libraries - **GC overhead** - Garbage collector - **Good optimization** - HotSpot JIT is mature **Implementation:** - Uses `BigInteger` class for precision - Machin's formula with JVM overhead - Large runtime (43.1 MB) #### 19. Python (47 ms, 9.7 MB) - Moderate **Why moderate:** - **Interpretation** - Bytecode execution - **Dynamic typing** - Runtime type checking - **GIL** - Global Interpreter Lock - **Large runtime** - Comprehensive standard library - **Good optimization** - CPython is well-optimized **Implementation:** - Uses `decimal` module for precision - Machin's formula with interpretation overhead - Moderate runtime (9.7 MB) #### 20. Perl (47 ms, 12.5 MB) - Moderate **Why moderate:** - **Interpretation overhead** - Line-by-line execution - **Dynamic typing** - Type checking at runtime - **Large runtime** - Comprehensive standard library - **Regex engine** - Powerful but overhead - **Mature** - Years of optimization **Implementation:** - Uses `Math::BigInt` module - Machin's formula with interpretation overhead - Large runtime (12.5 MB) ### Lower Performers (Rank 21-32) #### 21. Brainfuck (50 ms, 9.3 MB) - Moderate **Why moderate:** - **Minimal language** - Only 8 instructions - **Interpreter overhead** - Must interpret each instruction - **Simple operations** - No complex operations - **Small runtime** - Minimal interpreter - **Educational** - Not designed for performance **Implementation:** - Uses custom BigInt implementation - Machin's formula with interpretation overhead - Moderate runtime (9.3 MB) #### 22. Kotlin (60 ms, 45.2 MB) - Moderate **Why moderate:** - **JVM overhead** - Same as Java - **Kotlin runtime** - Additional Kotlin libraries - **JIT compilation** - Compiles to JVM bytecode - **GC overhead** - Garbage collector - **Good optimization** - Leverages JVM optimizations **Implementation:** - Uses `BigInteger` from Java stdlib - Machin's formula with JVM + Kotlin overhead - Large runtime (45.2 MB) #### 23. C# (66 ms, 41.5 MB) - Moderate **Why moderate:** - **CLR overhead** - Common Language Runtime - **JIT compilation** - Compiles IL to machine code - **Large runtime** - .NET framework included - **GC overhead** - Garbage collector - **Good optimization** - Mature JIT compiler **Implementation:** - Uses `System.Numerics.BigInteger` - Machin's formula with CLR overhead - Large runtime (41.5 MB) #### 24. PHP (68 ms, 26.5 MB) - Moderate **Why moderate:** - **Interpretation** - Bytecode execution - **Dynamic typing** - Runtime type checking - **Large runtime** - Web-focused standard library - **GC overhead** - Garbage collector - **Good for web** - Not designed for computation **Implementation:** - Uses `bcmath` extension for precision - Machin's formula with interpretation overhead - Large runtime (26.5 MB) #### 25. Ruby (79 ms, 28.8 MB) - Slower **Why slower:** - **Interpretation** - Bytecode execution - **Dynamic typing** - Runtime type checking - **Large runtime** - Object-oriented overhead - **GC overhead** - Garbage collector - **Less optimized** - Slower than Python **Implementation:** - Uses `BigDecimal` library - Machin's formula with interpretation overhead - Large runtime (28.8 MB) #### 26. JavaScript (89 ms, 44.8 MB) - Moderate **Why moderate:** - **JIT compilation** - V8 engine compiles to machine code - **Dynamic typing** - Runtime type checking - **Large runtime** - JavaScript engine overhead - **GC overhead** - Garbage collector - **Good optimization** - V8 is highly optimized **Implementation:** - Uses `BigInt` for arbitrary precision - Machin's formula with JIT overhead - Large runtime (44.8 MB) #### 27. Julia (157 ms, 235.9 MB) - Moderate **Why moderate:** - **JIT compilation** - Compiles to machine code - **Large runtime** - Scientific libraries included - **Dynamic typing** - Runtime type checking - **GC overhead** - Garbage collector - **Designed for science** - Optimized for numerical work **Implementation:** - Uses `BigFloat` for precision - Machin's formula with JIT overhead - Very large runtime (235.9 MB) #### 28. R (163 ms, 90.9 MB) - Moderate **Why moderate:** - **Interpretation** - R interpreter - **Dynamic typing** - Runtime type checking - **Large runtime** - Statistical libraries included - **GC overhead** - Garbage collector - **Designed for stats** - Not optimized for general computation **Implementation:** - Uses `Rmpfr` package for precision - Machin's formula with interpretation overhead - Large runtime (90.9 MB) #### 29. Erlang (176 ms, 77.4 MB) - Moderate **Why moderate:** - **BEAM VM** - Erlang virtual machine - **Functional overhead** - Immutability, pattern matching - **Concurrency focus** - Designed for distributed systems - **Large runtime** - BEAM VM included - **Not optimized for math** - Designed for reliability **Implementation:** - Uses `Integer` module for precision - Machin's formula with BEAM overhead - Large runtime (77.4 MB) #### 30. Scala (344 ms, 55.5 MB) - Slow **Why slow:** - **JVM overhead** - Same as Java - **Scala runtime** - Large standard library - **Functional overhead** - Immutability, pattern matching - **Complex compilation** - Advanced type system - **Less optimized** - More overhead than Java **Implementation:** - Uses `BigInt` from Scala library - Machin's formula with functional overhead - Very large runtime (55.5 MB) #### 31. Elixir (401 ms, 89.2 MB) - Slow **Why slow:** - **BEAM VM** - Erlang virtual machine - **Functional overhead** - Immutability, pattern matching - **Concurrency focus** - Designed for distributed systems - **Large runtime** - BEAM VM + Elixir libraries - **Not optimized for math** - Designed for reliability **Implementation:** - Uses `Integer` module for precision - Machin's formula with BEAM overhead - Very large runtime (89.2 MB) #### 32. TypeScript (931 ms, 218.9 MB) - Slowest **Why slow:** - **TypeScript compiler** - Extra compilation step - **JavaScript runtime** - V8 engine overhead - **Large runtime** - TypeScript + JavaScript libraries - **Dynamic typing** - Runtime type checking - **Not optimized** - Designed for web development **Implementation:** - Uses `BigInt` for precision - Machin's formula with TypeScript + JS overhead - Very large runtime (218.9 MB) ## Languages Tested **Compiled (10):** Assembly, C, C++, Rust, Go, Nim, Odin, Fortran, Swift, Crystal **JIT-Compiled (4):** Java, C#, Kotlin, Julia **Interpreted (5):** Python, Perl, PHP, Ruby, JavaScript **Other (15):** Bash, Brainfuck, D, Dart, Elixir, Erlang, Haskell, Lua, Objective-C, R, Scala, TypeScript, Vimscript, Wolfram, Zig ## Repository Structure ``` . ├── README.md # This file ├── reports/ # Detailed performance reports │ ├── summary.md # Overall summary │ ├── 1_decimals.md # 1 decimal precision │ ├── 2_decimals.md # 2 decimals precision │ ├── 5_decimals.md # 5 decimals precision │ ├── 10_decimals.md # 10 decimals precision │ ├── 100_decimals.md # 100 decimals precision │ ├── 1000_decimals.md # 1000 decimals precision │ └── 2000_decimals.md # 2000 decimals precision ├── timelines/ # Resource usage timeline data ├── assembly/ # Assembly implementation ├── c/ # C implementation ├── cpp/ # C++ implementation ├── rust/ # Rust implementation └── ... # Other language implementations ``` ## Running the Benchmark ```bash # Build all languages ./build.sh # Run all tests ./run_all.sh # Run specific language cd c && ./build.sh && ./print_hej # Run detailed profiling (breaks down execution time) ./profile_detailed.sh 100 ``` ## Detailed Profiling The benchmark includes a detailed profiling system that breaks down execution time into components: - **Startup Time**: Runtime initialization, library loading, JIT compilation - **Calculation Time**: Algorithm execution, numerical operations - **I/O Time**: Output formatting, result printing See [PROFILING.md](PROFILING.md) for detailed documentation. ## Additional Benchmark Metrics Beyond execution time and memory usage, there are many other metrics that can be valuable when benchmarking programming languages: ### System Resources #### CPU Metrics - **CPU Instructions**: Number of instructions executed - **CPU Cycles**: Clock cycles per instruction (CPI) - **Cache Misses**: L1, L2, L3 cache misses - **Branch Prediction**: Missed branch predictions - **CPU Cores**: Number of cores utilized #### Memory Metrics - **Virtual Memory**: Total allocated memory - **Memory Fragmentation**: Wasted memory due to fragmentation - **Memory Leaks**: Memory not properly freed - **Memory Patterns**: Allocation patterns over time - **Stack vs Heap**: Stack vs heap memory usage #### I/O Operations - **Disk Reads**: Bytes read from disk - **Disk Writes**: Bytes written to disk - **Network I/O**: Bytes sent/received - **File Handles**: Number of open files ### Performance Metrics #### Startup Time - **Cold Start**: Time to start program first time - **Warm Start**: Time to start program after cache - **JIT Warmup**: Time for JIT to optimize code #### Compilation Time - **Compilation**: Time to compile source code - **Linking**: Time to link binary - **Optimization**: Time for compiler optimizations #### Binary Size - **Binary Size**: Size of compiled binary - **Stripped Binary**: Size after stripping debug info - **Debug Info**: Size with debug information ### Quality Metrics #### Code Quality - **Lines of Code (LOC)**: Code lines - **Cyclomatic Complexity**: Code complexity - **Maintainability Index**: Maintainability - **Technical Debt**: Technical debt #### Security - **Memory Safety**: Buffer overflows, use-after-free - **Type Safety**: Type errors at compile/runtime - **Vulnerabilities**: Known security holes #### Portability - **Platforms**: Supported platforms - **Architectures**: x86, ARM, etc. - **Operating Systems**: Windows, Linux, macOS ### Developer Experience #### Development Environment - **IDE Support**: Syntax highlighting, autocomplete - **Debugging**: Step-by-step debugging - **Profiling**: Performance profiling - **Documentation**: Quality of documentation #### Community & Ecosystem - **Libraries**: Available libraries - **Package Managers**: npm, pip, cargo, etc. - **Community Size**: Number of developers - **Stack Overflow**: Questions/answers ### Cost Analysis #### Development Cost - **Learning Curve**: Time to learn language - **Productivity**: Code per time unit - **Debugging**: Time to find/fix bugs - **Maintenance**: Time to maintain code #### Runtime Cost - **Server Cost**: CPU, memory, disk - **Energy Cost**: Power consumption - **Licenses**: Cost for tools/libraries ### Scalability #### Performance Scaling - **Vertical Scaling**: Performance with more CPU/memory - **Horizontal Scaling**: Performance with more instances - **Concurrency**: Performance impact of parallelism #### Data Scaling - **Data Size**: Performance with more data - **Complexity**: Time complexity (O(n), O(n²), etc.) - **Memory Complexity**: Memory complexity ### Language-Specific Metrics #### Compiled Languages - **Compilation Time**: Time to compile - **Binary Size**: Size of binary - **Optimization Levels**: -O0, -O1, -O2, -O3 - **Linking Time**: Time to link #### JIT Languages - **JIT Warmup**: Time for JIT to optimize - **Deoptimization**: When JIT falls back - **GC Pauses**: Garbage collection pauses - **HotSpot Optimization**: JIT optimizations #### Interpreted Languages - **Interpretation Overhead**: Overhead for interpretation - **Bytecode Size**: Size of bytecode - **Dynamic Typing**: Overhead for type checking - **Eval Performance**: Performance of dynamic code ### Measurement Tools #### Linux ```bash # CPU performance perf stat -e cache-misses,cache-references ./program # Memory usage /usr/bin/time -v ./program 2>&1 | grep "Maximum resident" # Page faults /usr/bin/time -v ./program 2>&1 | grep "page faults" ``` #### macOS ```bash # CPU and memory /usr/bin/time -l ./program # Binary size ls -lh program strip program ls -lh program # Compilation time time gcc -O2 program.c -o program ``` #### Universal ```bash # Basic timing time ./program # Benchmarking hyperfine --warmup 3 './program' # Memory profiling valgrind --tool=massif ./program ``` ### Recommended Additions #### New Metrics to Add 1. **Binary Size** for all languages 2. **Compilation Time** for compiled languages 3. **Startup Time** (cold vs warm) 4. **CPU Instructions** (if possible) 5. **Cache Misses** (if possible) 6. **Page Faults** (if possible) 7. **Energy Consumption** (if possible) #### New Charts to Create 1. **Binary Size vs Performance** 2. **Compilation Time vs Performance** 3. **Memory Usage Over Time** 4. **CPU Usage Over Time** 5. **Performance Scaling with Decimals** ### Example Measurements #### Binary Size Comparison ``` Language Binary Size (KB) Stripped (KB) Assembly 16 12 C 824 612 C++ 1,024 768 Rust 1,536 1,024 Go 2,048 1,536 Java N/A (JVM) N/A Python N/A (interpreted) N/A ``` #### Compilation Time ``` Language Compilation Time (ms) C 50 C++ 75 Rust 200 Go 150 Java 100 ``` #### Startup Time ``` Language Cold Start (ms) Warm Start (ms) C 1 1 Java 150 50 Python 30 10 Node.js 100 30 ``` --- **Note**: This benchmark currently measures execution time and memory usage. Adding these additional metrics would provide a more comprehensive view of language performance, but would require additional tooling and measurement infrastructure. ## License MIT License - See LICENSE file for details. --- *Generated from Pi Calculation Benchmark - Apple A18 Pro Performance Study*