Hi, I ran the following experiment, and I was wondering if someone can explain the results. 1. I cloned the v23.1.0.0 release and built the x86 gem5.fast binary. 2. I then built the SPEC CPU 2017 disk image using the instructions in the documentation. 3. In my config file, I use a 1-core KVM CPU to boot into Linux in FS mode, then switch CPUs to a 1-core O3 CPU, run the benchmark for 10M instructions, reset stats, continue for 1M instructions then dump stats and exit. This is roughly based on the x86 spec17 config file in `configs/example/gem5_library/`. Now, when I run this twice, I get completely different numbers in stats.txt. This is unexpected to me because the stats are for the 1M instructions run using the O3 CPU, and as far as I understand, this should be deterministic. The O3 CPU itself should be deterministic AND the executed instructions themselves should be identical. This doesn't seem to be the case, however, so I was wondering if someone could point out where my assumptions fail (or if I'm doing something incorrectly in the experiment). I would also greatly appreciate it if someone could point me to how to make benchmarking robust because the numbers I get vary quite a lot across runs. Thanks a lot. Best, Hossam

1. I cloned the v23.1.0.0 release and built the x86 gem5.fast binary.
2. I then built the SPEC CPU 2017 disk image using the instructions in
the documentation.
3. In my config file, I use a 1-core KVM CPU to boot into Linux in FS
mode, then switch CPUs to a 1-core O3 CPU, run the benchmark for 10M
instructions, reset stats, continue for 1M instructions then dump stats and
exit. This is roughly based on the x86 spec17 config file in
`configs/example/gem5_library/`.

Hi Hossam, When using the KVM CPU you won't get deterministic results. Since the KVM CPU is using your *host* processor to execute the instructions (1) there is no determinism across hosts and (2) there's no API to get instruction-perfect simulation intervals. If you use the atomic CPU for fast-forwarding to the ROI, you should get deterministic results. Another option is to use KVM to fast forward to the ROI and then *take a checkpoint*. Then, restoring from the checkpoint (in atomic or timing mode) should give you deterministic results. Finally, the other option is to use something like SMARTS-based sampled simulation and run the experiment multiple times and take an average. Cheers, Jason On Fri, Nov 1, 2024 at 8:05 AM Hossam ElAtali via gem5-users < gem5-users@gem5.org> wrote: > Hi, > > I ran the following experiment, and I was wondering if someone can explain > the results. > > > 1. I cloned the v23.1.0.0 release and built the x86 gem5.fast binary. > 2. I then built the SPEC CPU 2017 disk image using the instructions in > the documentation. > 3. In my config file, I use a 1-core KVM CPU to boot into Linux in FS > mode, then switch CPUs to a 1-core O3 CPU, run the benchmark for 10M > instructions, reset stats, continue for 1M instructions then dump stats and > exit. This is roughly based on the x86 spec17 config file in > `configs/example/gem5_library/`. > > > Now, when I run this twice, I get completely different numbers in > stats.txt. This is unexpected to me because the stats are for the 1M > instructions run using the O3 CPU, and as far as I understand, this should > be deterministic. The O3 CPU itself should be deterministic AND the > executed instructions themselves should be identical. This doesn't seem > to be the case, however, so I was wondering if someone could point out > where my assumptions fail (or if I'm doing something incorrectly in the > experiment). I would also greatly appreciate it if someone could point me > to how to make benchmarking robust because the numbers I get vary quite a > lot across runs. Thanks a lot. > > Best, > Hossam > _______________________________________________ > gem5-users mailing list -- gem5-users@gem5.org > To unsubscribe send an email to gem5-users-leave@gem5.org >