praca_magisterska/scripts/test_scenarios.md

# Test Scenarios Document
**Project**: Master's Thesis - Game Engine Performance Comparison  
**Date**: January 19, 2026  
**Author**: Krzysztof Rudnicki  

---

## Overview

This document defines the test scenarios for comparing Unity and Unreal Engine performance using bullet-hell games and NVIDIA Nsight profiling.

---

## Test Hardware

- **CPU**: AMD Ryzen 9 7900X3D 12-Core Processor (24 cores, 48 threads)
- **GPU**: NVIDIA GeForce RTX 3090 (24 GB GDDR6X)
- **RAM**: 32 GB
- **OS**: Arch Linux (kernel 6.18.5-arch1-1)
- **NVIDIA Drivers**: 590.48.01

---

## Test Methodology

### Game: Bullet-Hell (Unity & Unreal implementations)

Both games implement identical gameplay mechanics:
- Player-controlled ship
- Enemy spawning system with escalating load
- Bullet pattern generation
- Collision detection
- 90-second survival mode with progressive load increase

> **Note**: The game runs continuously for 90 seconds with three load phases:
> - **0-30 seconds**: Low load phase
> - **30-60 seconds**: Medium load phase  
> - **60-90 seconds**: High load phase

### Performance Metrics to Capture

1. **Frame Time** (ms) - Time to render one frame
2. **FPS** (frames per second) - Derived from frame time
3. **GPU Utilization** (%) - Percentage of GPU capacity used
4. **Memory Usage** (MB) - VRAM and system RAM consumption
5. **Draw Calls** - Number of draw calls per frame
6. **Vertex Count** - Total vertices rendered per frame

---

## Load Phases

### Phase 1: Low Load (0-30 seconds)
**Objective**: Establish baseline performance with minimal load

**Parameters**:
- **Bullet count on screen**: 50-100 bullets
- **Active enemies**: 2-3 enemies
- **Time window**: 0-30 seconds of gameplay

**Expected Outcomes**:
- Stable frame rate (60 FPS target)
- Low GPU utilization (<50%)
- Minimal memory usage

**Nsight Capture Points**:
- 10 seconds (early low load)
- 20 seconds (stable low load)
- 25 seconds (end of low load phase)

---

### Phase 2: Medium Load (30-60 seconds)
**Objective**: Test performance under moderate load

**Parameters**:
- **Bullet count on screen**: 200-300 bullets
- **Active enemies**: 5-7 enemies
- **Time window**: 30-60 seconds of gameplay

**Expected Outcomes**:
- Moderate GPU utilization (50-70%)
- Frame rate may drop slightly
- Increased memory usage

**Nsight Capture Points**:
- 35 seconds (transition to medium load)
- 45 seconds (stable medium load)
- 55 seconds (end of medium load phase)

---

### Phase 3: High Load (60-90 seconds)
**Objective**: Test performance under maximum load (stress test)

**Parameters**:
- **Bullet count on screen**: 500+ bullets
- **Active enemies**: 10+ enemies
- **Time window**: 60-90 seconds of gameplay

**Expected Outcomes**:
- High GPU utilization (>70%)
- Potential frame drops
- Maximum memory usage observed

**Nsight Capture Points**:
- 65 seconds (transition to high load)
- 75 seconds (peak sustained load)
- 85 seconds (end of high load phase)

---

## Data Collection Procedure

### For Each Test Run:

1. **Pre-test Setup**:
   - Run `./scripts/close_background_apps.sh` to close unnecessary applications
   - Verify system is idle

2. **Unity Testing**:
   - Start NVIDIA Nsight Systems (`nsys-ui`)
   - Configure Nsight to profile the Unity game executable
   - Launch Unity bullet-hell game through Nsight
   - Let the game run for full 90 seconds (all three load phases)
   - Nsight captures performance data throughout the entire run
   - Save capture data with naming: `unity_run_[N].nsight-rep`
   - Export metrics to CSV: `unity_metrics_[N].csv`

3. **Unreal Testing**:
   - Close Unity completely
   - Start NVIDIA Nsight Systems (`nsys-ui`)
   - Configure Nsight to profile the Unreal game executable
   - Launch Unreal bullet-hell game through Nsight
   - Let the game run for full 90 seconds (all three load phases)
   - Nsight captures performance data throughout the entire run
   - Save capture data with naming: `unreal_run_[N].nsight-rep`
   - Export metrics to CSV: `unreal_metrics_[N].csv`

4. **Post-test Data Organization**:
   - Organize captures in folders: `data/nsight/unity/` and `data/nsight/unreal/`
   - Compile metrics into master spreadsheet: `performance_comparison.xlsx`
   - Take screenshots of key Nsight analysis views
   - Document any anomalies or issues observed

---

## Expected Data Outputs

### Per Test Run (3 runs per engine, 6 total):
- Nsight capture file (.nsight-rep)
- Metrics CSV with frame time, FPS, GPU %, memory, draw calls, vertices
- Screenshot of Nsight GPU trace for each load phase
- Screenshot of Nsight memory analysis

### Aggregate Data:
- Comparison table: Unity vs Unreal per load phase
- Performance graphs showing all metrics over 90-second timeline
- Statistical analysis (mean, std dev, min, max) per load phase

---

## Quality Assurance

### Validation Checks:
- [ ] All three runs completed for Unity
- [ ] All three runs completed for Unreal
- [ ] All CSV metrics exported
- [ ] Screenshots saved for all load phases
- [ ] Data organized in proper folder structure
- [ ] No system crashes or anomalies during testing
- [ ] Comparable conditions between Unity and Unreal runs

### Data Integrity:
- [ ] Frame times make sense (>0ms, <100ms typically)
- [ ] GPU utilization in valid range (0-100%)
- [ ] Memory values reasonable for game scope
- [ ] No obvious outliers without explanation

---

## Timeline

- **Day 1 (Sunday, Jan 19)**: Setup Nsight, run first test on Unity engine
- **Day 2 (Monday, Jan 20)**: Complete Unity tests (3 runs), run first test on Unreal engine
- **Day 3 (Tuesday, Jan 21)**: Complete Unreal tests (3 runs), verify data integrity
- **Day 4 (Wednesday, Jan 22)**: Analyze results, create comparison graphs, organize data

---

## Notes

- If a capture fails, note the reason and retry
- Document any differences in how load phases manifest in Unity vs Unreal
- Pay attention to engine-specific optimizations that may affect results
- Consider recording video of test runs for reference

---

**Status**: Ready for execution  
**Next Action**: Run `./scripts/close_background_apps.sh` and start Nsight profiling