praca_magisterska/scripts/test_scenarios.md

Test Scenarios Document

Project: Master's Thesis - Game Engine Performance Comparison
Date: January 16, 2026
Author: Krzysztof Rudnicki

---

Overview

This document defines the test scenarios for comparing Unity and Unreal Engine performance using bullet-hell games and NVIDIA Nsight profiling.

---

Test Hardware

[Auto-generated by running: ./scripts/get_hardware_specs.sh]

• CPU: [Your CPU model]
• GPU: [Your GPU model]
• RAM: [Your RAM amount]
• OS: [Your OS version]

---

Test Methodology

Game: Bullet-Hell (Unity & Unreal implementations)

Both games implement identical gameplay mechanics:

• Player-controlled ship
• Enemy spawning system with escalating difficulty
• Bullet pattern generation
• Collision detection
• 90-second survival mode

Performance Metrics to Capture

1. Frame Time (ms) - Time to render one frame
2. FPS (frames per second) - Derived from frame time
3. GPU Utilization (%) - Percentage of GPU capacity used
4. Memory Usage (MB) - VRAM and system RAM consumption
5. Draw Calls - Number of draw calls per frame
6. Vertex Count - Total vertices rendered per frame

---

Test Scenarios

Scenario 1: Low Difficulty (Baseline)

Objective: Establish baseline performance with minimal load

Parameters:

• Bullet count on screen: 50-100 bullets
• Active enemies: 2-3 enemies
• Duration: 30 seconds of gameplay
• Captures: 5 frame captures at different moments

Expected Outcomes:

• Stable frame rate (60 FPS target)
• Low GPU utilization (<50%)
• Minimal memory usage

Nsight Capture Points:

• 5 seconds into scenario
• 10 seconds into scenario
• 15 seconds into scenario
• 20 seconds into scenario
• 25 seconds into scenario

---

Scenario 2: Medium Difficulty

Objective: Test performance under moderate load

Parameters:

• Bullet count on screen: 200-300 bullets
• Active enemies: 5-7 enemies
• Duration: 30 seconds of gameplay
• Captures: 5 frame captures at different moments

Expected Outcomes:

• Moderate GPU utilization (50-70%)
• Frame rate may drop slightly
• Increased memory usage

Nsight Capture Points:

• 5 seconds into scenario
• 10 seconds into scenario
• 15 seconds into scenario
• 20 seconds into scenario
• 25 seconds into scenario

---

Scenario 3: High Difficulty (Stress Test)

Objective: Test performance under maximum load

Parameters:

• Bullet count on screen: 500+ bullets
• Active enemies: 10+ enemies
• Duration: 30 seconds of gameplay
• Captures: 5 frame captures at different moments

Expected Outcomes:

• High GPU utilization (>70%)
• Potential frame drops
• Maximum memory usage observed

Nsight Capture Points:

• 5 seconds into scenario (system stabilizing)
• 10 seconds into scenario (peak load)
• 15 seconds into scenario (sustained load)
• 20 seconds into scenario (peak load)
• 25 seconds into scenario (sustained load)

---

Data Collection Procedure

For Each Scenario:

1. Pre-test Setup:

   • Close all background applications
   • Disable system updates
   • Set power mode to "High Performance"
   • Wait 5 minutes for system stabilization
   • Clear GPU/system caches

2. Unity Testing:

   • Launch Unity bullet-hell game
   • Navigate to scenario (Low/Medium/High)
   • Start NVIDIA Nsight Graphics
   • Attach Nsight to Unity process
   • Play scenario for 30 seconds
   • Capture frames at designated time points (5 captures)
   • Save capture data with naming: unity_[scenario]_capture_[N].nsight
   • Export metrics to CSV: unity_[scenario]_metrics.csv

3. Unreal Testing:

   • Close Unity completely
   • Wait 2 minutes for system cooldown
   • Launch Unreal bullet-hell game
   • Navigate to scenario (Low/Medium/High)
   • Start NVIDIA Nsight Graphics
   • Attach Nsight to Unreal process
   • Play scenario for 30 seconds
   • Capture frames at designated time points (5 captures)
   • Save capture data with naming: unreal_[scenario]_capture_[N].nsight
   • Export metrics to CSV: unreal_[scenario]_metrics.csv

4. Post-test Data Organization:

   • Organize captures in folders: data/nsight/unity/ and data/nsight/unreal/
   • Compile metrics into master spreadsheet: performance_comparison.xlsx
   • Take screenshots of key Nsight analysis views
   • Document any anomalies or issues observed

---

Expected Data Outputs

Per Test Run (15 total: 3 scenarios × 5 captures):

• Nsight capture file (.nsight)
• Metrics CSV with frame time, FPS, GPU %, memory, draw calls, vertices
• Screenshot of Nsight GPU trace
• Screenshot of Nsight memory analysis

Aggregate Data:

• Comparison table: Unity vs Unreal per scenario
• Performance graphs showing all metrics
• Statistical analysis (mean, std dev, min, max)

---

Quality Assurance

Validation Checks:

• All scenarios tested in both engines
• 5 captures per scenario completed
• All CSV metrics exported
• Screenshots saved for all captures
• Data organized in proper folder structure
• No system crashes or anomalies during testing
• Comparable conditions between Unity and Unreal runs

Data Integrity:

• Frame times make sense (>0ms, <100ms typically)
• GPU utilization in valid range (0-100%)
• Memory values reasonable for game scope
• No obvious outliers without explanation

---

Timeline

• Day 1 (Thursday, Jan 16): Setup Nsight, run Scenario 1 on both engines
• Day 2 (Friday, Jan 17): Run Scenario 2 on both engines
• Day 3 (Saturday, Jan 18): Run Scenario 3 on both engines
• Day 4 (Sunday, Jan 19): Verify data, re-run any problematic captures, organize data

---

Notes

• If a capture fails, note the reason and retry
• Document any differences in how scenarios are triggered in Unity vs Unreal
• Pay attention to engine-specific optimizations that may affect results
• Consider recording video of test runs for reference

---

Status: Ready for execution
Next Action: Run ./scripts/get_hardware_specs.sh to generate hardware specs for Chapter 4