praca_magisterska/scripts/test_scenarios.md

# Test Scenarios Document
**Project**: Master's Thesis - Game Engine Performance Comparison  
**Date**: January 16, 2026  
**Author**: Krzysztof Rudnicki  

---

## Overview

This document defines the test scenarios for comparing Unity and Unreal Engine performance using bullet-hell games and NVIDIA Nsight profiling.

---

## Test Hardware

*[Auto-generated by running: `./scripts/get_hardware_specs.sh`]*

- **CPU**: [Your CPU model]
- **GPU**: [Your GPU model]
- **RAM**: [Your RAM amount]
- **OS**: [Your OS version]

---

## Test Methodology

### Game: Bullet-Hell (Unity & Unreal implementations)

Both games implement identical gameplay mechanics:
- Player-controlled ship
- Enemy spawning system with escalating difficulty
- Bullet pattern generation
- Collision detection
- 90-second survival mode

### Performance Metrics to Capture

1. **Frame Time** (ms) - Time to render one frame
2. **FPS** (frames per second) - Derived from frame time
3. **GPU Utilization** (%) - Percentage of GPU capacity used
4. **Memory Usage** (MB) - VRAM and system RAM consumption
5. **Draw Calls** - Number of draw calls per frame
6. **Vertex Count** - Total vertices rendered per frame

---

## Test Scenarios

### Scenario 1: Low Difficulty (Baseline)
**Objective**: Establish baseline performance with minimal load

**Parameters**:
- **Bullet count on screen**: 50-100 bullets
- **Active enemies**: 2-3 enemies
- **Duration**: 30 seconds of gameplay
- **Captures**: 5 frame captures at different moments

**Expected Outcomes**:
- Stable frame rate (60 FPS target)
- Low GPU utilization (<50%)
- Minimal memory usage

**Nsight Capture Points**:
- 5 seconds into scenario
- 10 seconds into scenario
- 15 seconds into scenario
- 20 seconds into scenario
- 25 seconds into scenario

---

### Scenario 2: Medium Difficulty
**Objective**: Test performance under moderate load

**Parameters**:
- **Bullet count on screen**: 200-300 bullets
- **Active enemies**: 5-7 enemies
- **Duration**: 30 seconds of gameplay
- **Captures**: 5 frame captures at different moments

**Expected Outcomes**:
- Moderate GPU utilization (50-70%)
- Frame rate may drop slightly
- Increased memory usage

**Nsight Capture Points**:
- 5 seconds into scenario
- 10 seconds into scenario
- 15 seconds into scenario
- 20 seconds into scenario
- 25 seconds into scenario

---

### Scenario 3: High Difficulty (Stress Test)
**Objective**: Test performance under maximum load

**Parameters**:
- **Bullet count on screen**: 500+ bullets
- **Active enemies**: 10+ enemies
- **Duration**: 30 seconds of gameplay
- **Captures**: 5 frame captures at different moments

**Expected Outcomes**:
- High GPU utilization (>70%)
- Potential frame drops
- Maximum memory usage observed

**Nsight Capture Points**:
- 5 seconds into scenario (system stabilizing)
- 10 seconds into scenario (peak load)
- 15 seconds into scenario (sustained load)
- 20 seconds into scenario (peak load)
- 25 seconds into scenario (sustained load)

---

## Data Collection Procedure

### For Each Scenario:

1. **Pre-test Setup**:
   - Close all background applications
   - Disable system updates
   - Set power mode to "High Performance"
   - Wait 5 minutes for system stabilization
   - Clear GPU/system caches

2. **Unity Testing**:
   - Launch Unity bullet-hell game
   - Navigate to scenario (Low/Medium/High)
   - Start NVIDIA Nsight Graphics
   - Attach Nsight to Unity process
   - Play scenario for 30 seconds
   - Capture frames at designated time points (5 captures)
   - Save capture data with naming: `unity_[scenario]_capture_[N].nsight`
   - Export metrics to CSV: `unity_[scenario]_metrics.csv`

3. **Unreal Testing**:
   - Close Unity completely
   - Wait 2 minutes for system cooldown
   - Launch Unreal bullet-hell game
   - Navigate to scenario (Low/Medium/High)
   - Start NVIDIA Nsight Graphics
   - Attach Nsight to Unreal process
   - Play scenario for 30 seconds
   - Capture frames at designated time points (5 captures)
   - Save capture data with naming: `unreal_[scenario]_capture_[N].nsight`
   - Export metrics to CSV: `unreal_[scenario]_metrics.csv`

4. **Post-test Data Organization**:
   - Organize captures in folders: `data/nsight/unity/` and `data/nsight/unreal/`
   - Compile metrics into master spreadsheet: `performance_comparison.xlsx`
   - Take screenshots of key Nsight analysis views
   - Document any anomalies or issues observed

---

## Expected Data Outputs

### Per Test Run (15 total: 3 scenarios × 5 captures):
- Nsight capture file (.nsight)
- Metrics CSV with frame time, FPS, GPU %, memory, draw calls, vertices
- Screenshot of Nsight GPU trace
- Screenshot of Nsight memory analysis

### Aggregate Data:
- Comparison table: Unity vs Unreal per scenario
- Performance graphs showing all metrics
- Statistical analysis (mean, std dev, min, max)

---

## Quality Assurance

### Validation Checks:
- [ ] All scenarios tested in both engines
- [ ] 5 captures per scenario completed
- [ ] All CSV metrics exported
- [ ] Screenshots saved for all captures
- [ ] Data organized in proper folder structure
- [ ] No system crashes or anomalies during testing
- [ ] Comparable conditions between Unity and Unreal runs

### Data Integrity:
- [ ] Frame times make sense (>0ms, <100ms typically)
- [ ] GPU utilization in valid range (0-100%)
- [ ] Memory values reasonable for game scope
- [ ] No obvious outliers without explanation

---

## Timeline

- **Day 1 (Thursday, Jan 16)**: Setup Nsight, run Scenario 1 on both engines
- **Day 2 (Friday, Jan 17)**: Run Scenario 2 on both engines
- **Day 3 (Saturday, Jan 18)**: Run Scenario 3 on both engines
- **Day 4 (Sunday, Jan 19)**: Verify data, re-run any problematic captures, organize data

---

## Notes

- If a capture fails, note the reason and retry
- Document any differences in how scenarios are triggered in Unity vs Unreal
- Pay attention to engine-specific optimizations that may affect results
- Consider recording video of test runs for reference

---

**Status**: Ready for execution  
**Next Action**: Run `./scripts/get_hardware_specs.sh` to generate hardware specs for Chapter 4