Empathy List Archives

Άγγελος Καβαλέρος

Fri, Oct 11, 2024 5:49 PM

Greetings dear gem5 Community,

The problem I encounter is related with my inability to explain the
output of the metrics that heterogarnet provided.

The environment that I work in WSL(Windows Subsystem for Linux) from VSCode.

step_1)
As written clearly inside the:
https://www.gem5.org/documentation/learning_gem5/part1/building/
using the command :
python3 which scons build/NULL/gem5.opt -j3

step_2)
and after that from :
https://www.gem5.org/documentation/general_docs/ruby/garnet_synthetic_traffic/
using the following:
scons defconfig build/NULL build_opts/NULL
scons setconfig build/NULL RUBY_PROTOCOL_GARNET_STANDALONE=y
scons build/NULL/gem5.debug

step_3)
and I also made some changes inside
"/gem5/src/mem/ruby/network/garnet/GarnetNetwork.py"
altering (the lines 47 and 48) buffers_per_data_vc and
buffers_per_ctrl_vc to 8 before step_2, to change the data and ctrl
buffer size.

The environment is ready.

Then with an automated script I run the command below with injection
rate form 0.01 to 0.15 with packet length 1024 (the script stops when
latency is above 100 cycles):

./build/NULL/gem5.debug configs/example/garnet_synth_traffic.py
--num-cpus=64
--num-dirs=64
--network=garnet
--topology=Mesh_XY
--mesh-rows=8
--sim-cycles=100000000
--synthetic=uniform_random
--inj-vne=2
--injectionrate=$injectionrate
--router-latency=1
--link-latency=1
--vcs-per-vnet=1
--routing-algorithm=1
--sys-clock=1GHz
--ruby-clock=1GHz
--link-width-bits=1024

The problem is that for uniform random traffic in an 8x8 2D mesh with
routing (XY) algorithm, the saturation throughput flits/node/cycle that
i get is much less than the expected one from related papers ( my
network has a congestion too early!). For example i have attached some
pictures:

Any commend would be quite helpful. I will also provide the metrics in
excel for better understanding

Thank you in advance!

Greetings dear gem5 Community, The problem I encounter is related with my inability to explain the output of the metrics that heterogarnet provided. The environment that I work in WSL(Windows Subsystem for Linux) from VSCode. step_1) As written clearly inside the: https://www.gem5.org/documentation/learning_gem5/part1/building/ using the command : python3 which scons build/NULL/gem5.opt -j3 step_2) and after that from : https://www.gem5.org/documentation/general_docs/ruby/garnet_synthetic_traffic/ using the following: scons defconfig build/NULL build_opts/NULL scons setconfig build/NULL RUBY_PROTOCOL_GARNET_STANDALONE=y scons build/NULL/gem5.debug step_3) and I also made some changes inside "/gem5/src/mem/ruby/network/garnet/GarnetNetwork.py" altering (the lines 47 and 48) buffers_per_data_vc and buffers_per_ctrl_vc to 8 before step_2, to change the data and ctrl buffer size. The environment is ready. Then with an automated script I run the command below with injection rate form 0.01 to 0.15 with packet length 1024 (the script stops when latency is above 100 cycles): ./build/NULL/gem5.debug configs/example/garnet_synth_traffic.py --num-cpus=64 --num-dirs=64 --network=garnet --topology=Mesh_XY --mesh-rows=8 --sim-cycles=100000000 --synthetic=uniform_random --inj-vne=2 --injectionrate=$injectionrate --router-latency=1 --link-latency=1 --vcs-per-vnet=1 --routing-algorithm=1 --sys-clock=1GHz --ruby-clock=1GHz --link-width-bits=1024 The problem is that for uniform random traffic in an 8x8 2D mesh with routing (XY) algorithm, the saturation throughput flits/node/cycle that i get is much less than the expected one from related papers ( my network has a congestion too early!). For example i have attached some pictures: Any commend would be quite helpful. I will also provide the metrics in excel for better understanding Thank you in advance!

KT

Krishna, Tushar

Fri, Oct 11, 2024 8:11 PM

Try with --inj-vnet=0 (ie single flit pavkets) and vcs-per-vnet=8 first (such that sufficient number of VCs) ..
Before going to multi-flit packets (inj-vnet=2)

And then with multi-flit packets also try more VCs .. that may be the reason for lower throughput than you anticipate.
From my recollection of garnet stats, you get about 0.4 flits/node/cycle of saturation throughput for 8x8 mesh (recall that theoretical max for 8x8 mesh is 0.5 flits/node/cycle) with single flit packets.

Cheers,
Tushar
On Oct 11, 2024 at 1:56 PM -0400, Άγγελος Καβαλέρος via gem5-users gem5-users@gem5.org, wrote:
Greetings dear gem5 Community,

The problem I encounter is related with my inability to explain the
output of the metrics that heterogarnet provided.

The environment that I work in WSL(Windows Subsystem for Linux) from VSCode.

step_1)
As written clearly inside the:
https://www.gem5.org/documentation/learning_gem5/part1/building/
using the command :
python3 which scons build/NULL/gem5.opt -j3

step_2)
and after that from :
https://www.gem5.org/documentation/general_docs/ruby/garnet_synthetic_traffic/
using the following:
scons defconfig build/NULL build_opts/NULL
scons setconfig build/NULL RUBY_PROTOCOL_GARNET_STANDALONE=y
scons build/NULL/gem5.debug

step_3)
and I also made some changes inside
"/gem5/src/mem/ruby/network/garnet/GarnetNetwork.py"
altering (the lines 47 and 48) buffers_per_data_vc and
buffers_per_ctrl_vc to 8 before step_2, to change the data and ctrl
buffer size.

The environment is ready.

Then with an automated script I run the command below with injection
rate form 0.01 to 0.15 with packet length 1024 (the script stops when
latency is above 100 cycles):

./build/NULL/gem5.debug configs/example/garnet_synth_traffic.py
--num-cpus=64
--num-dirs=64
--network=garnet
--topology=Mesh_XY
--mesh-rows=8
--sim-cycles=100000000
--synthetic=uniform_random
--inj-vne=2
--injectionrate=$injectionrate
--router-latency=1
--link-latency=1
--vcs-per-vnet=1
--routing-algorithm=1
--sys-clock=1GHz
--ruby-clock=1GHz
--link-width-bits=1024

The problem is that for uniform random traffic in an 8x8 2D mesh with
routing (XY) algorithm, the saturation throughput flits/node/cycle that
i get is much less than the expected one from related papers ( my
network has a congestion too early!). For example i have attached some
pictures:

Any commend would be quite helpful. I will also provide the metrics in
excel for better understanding

Thank you in advance!

Try with --inj-vnet=0 (ie single flit pavkets) and vcs-per-vnet=8 first (such that sufficient number of VCs) .. Before going to multi-flit packets (inj-vnet=2) And then with multi-flit packets also try more VCs .. that may be the reason for lower throughput than you anticipate. From my recollection of garnet stats, you get about 0.4 flits/node/cycle of saturation throughput for 8x8 mesh (recall that theoretical max for 8x8 mesh is 0.5 flits/node/cycle) with single flit packets. Cheers, Tushar On Oct 11, 2024 at 1:56 PM -0400, Άγγελος Καβαλέρος via gem5-users <gem5-users@gem5.org>, wrote: Greetings dear gem5 Community, The problem I encounter is related with my inability to explain the output of the metrics that heterogarnet provided. The environment that I work in WSL(Windows Subsystem for Linux) from VSCode. step_1) As written clearly inside the: https://www.gem5.org/documentation/learning_gem5/part1/building/ using the command : python3 which scons build/NULL/gem5.opt -j3 step_2) and after that from : https://www.gem5.org/documentation/general_docs/ruby/garnet_synthetic_traffic/ using the following: scons defconfig build/NULL build_opts/NULL scons setconfig build/NULL RUBY_PROTOCOL_GARNET_STANDALONE=y scons build/NULL/gem5.debug step_3) and I also made some changes inside "/gem5/src/mem/ruby/network/garnet/GarnetNetwork.py" altering (the lines 47 and 48) buffers_per_data_vc and buffers_per_ctrl_vc to 8 before step_2, to change the data and ctrl buffer size. The environment is ready. Then with an automated script I run the command below with injection rate form 0.01 to 0.15 with packet length 1024 (the script stops when latency is above 100 cycles): ./build/NULL/gem5.debug configs/example/garnet_synth_traffic.py --num-cpus=64 --num-dirs=64 --network=garnet --topology=Mesh_XY --mesh-rows=8 --sim-cycles=100000000 --synthetic=uniform_random --inj-vne=2 --injectionrate=$injectionrate --router-latency=1 --link-latency=1 --vcs-per-vnet=1 --routing-algorithm=1 --sys-clock=1GHz --ruby-clock=1GHz --link-width-bits=1024 The problem is that for uniform random traffic in an 8x8 2D mesh with routing (XY) algorithm, the saturation throughput flits/node/cycle that i get is much less than the expected one from related papers ( my network has a congestion too early!). For example i have attached some pictures: Any commend would be quite helpful. I will also provide the metrics in excel for better understanding Thank you in advance! <Screenshot 2024-10-11 204829.png><Screenshot 2024-10-11 204757.png><Screenshot 2024-09-29 at 5.02.12 PM.png>

A

akavaler@ee.duth.gr

Fri, Oct 11, 2024 8:34 PM

Hello Krishna,
Thank you for the fast respond! The paper that i was studying was talking about 1 filt packets with 1 vc-per-vnet (wormhole) and as the metrics that i collected are telling me (i dont know why the photos i provided are not visible) i already have 1 flit packets ( FlitsIjected == PacketsInjected , so the link-width-bits = 1024 is more than enough). Form the paper the congestion should start happening above 0.075 packets/node/cycle and not at 0.065 as garnet is showning me here. Thats the problem….

uniform_randomwith1vcand1flit:

InjectionRate AverageFlitQueueingLatency AverageFlitNetworkLatency AverageFlitLatency AveragePacketQueueingLatency AveragePacketNetworkLatency AveragePacketLatency FlitsInjected FlitsReceived PacketsInjected PacketsReceived AverageHops

0.01 2110.250345 13836.06185 15946.3122 2110.250345 13836.06185 15946.3122 127677 127664 127677 127664 5.249366

0.015 2176.55784 14051.43619 16227.99403 2176.55784 14051.43619 16227.99403 192156 192141 192156 192141 5.255026

0.02 2251.20415 14289.92414 16541.12829 2251.20415 14289.92414 16541.12829 256420 256405 256420 256405 5.255424

0.025 2327.898415 14549.2637 16877.16211 2327.898415 14549.2637 16877.16211 320543 320520 320543 320520 5.251703

0.03 2427.341709 14874.98048 17302.32219 2427.341709 14874.98048 17302.32219 384328 384292 384328 384292 5.249984

0.035 2545.057922 15269.61109 17814.66901 2545.057922 15269.61109 17814.66901 448141 448101 448141 448101 5.249924

0.04 2707.391898 15755.96321 18463.35511 2707.391898 15755.96321 18463.35511 511800 511763 511800 511763 5.247974

0.045 2938.631156 16421.48544 19360.11659 2938.631156 16421.48544 19360.11659 575744 575683 575744 575683 5.25208

0.05 3354.394007 17314.71256 20669.10657 3354.394007 17314.71256 20669.10657 639705 639644 639705 639644 5.250088

0.055 4301.373952 18705.94991 23007.32386 4301.373952 18705.94991 23007.32386 703661 703591 703661 703591 5.25145

0.06 7323.253138 20918.07593 28241.32906 7323.253138 20918.07593 28241.32906 767327 767247 767327 767247 5.247669

0.065 33369.62551 25298.39128 58668.01679 33369.62551 25298.39128 58668.01679 831131 831033 831131 831033 5.251095

0.07 2658109.712 29815.79864 2687925.51 2658109.712 29815.79864 2687925.51 869634 869505 869634 869505 5.223994

thank you!

Hello Krishna,\ Thank you for the fast respond! The paper that i was studying was talking about 1 filt packets with 1 vc-per-vnet (wormhole) and as the metrics that i collected are telling me (i dont know why the photos i provided are not visible) i already have 1 flit packets ( FlitsIjected == PacketsInjected , so the link-width-bits = 1024 is more than enough). Form the paper the congestion should start happening above 0.075 packets/node/cycle and not at 0.065 as garnet is showning me here. Thats the problem…. uniform_randomwith1vcand1flit: InjectionRate AverageFlitQueueingLatency AverageFlitNetworkLatency AverageFlitLatency AveragePacketQueueingLatency AveragePacketNetworkLatency AveragePacketLatency FlitsInjected FlitsReceived PacketsInjected PacketsReceived AverageHops 0\.01 2110.250345 13836.06185 15946.3122 2110.250345 13836.06185 15946.3122 127677 127664 127677 127664 5.249366 0\.015 2176.55784 14051.43619 16227.99403 2176.55784 14051.43619 16227.99403 192156 192141 192156 192141 5.255026 0\.02 2251.20415 14289.92414 16541.12829 2251.20415 14289.92414 16541.12829 256420 256405 256420 256405 5.255424 0\.025 2327.898415 14549.2637 16877.16211 2327.898415 14549.2637 16877.16211 320543 320520 320543 320520 5.251703 0\.03 2427.341709 14874.98048 17302.32219 2427.341709 14874.98048 17302.32219 384328 384292 384328 384292 5.249984 0\.035 2545.057922 15269.61109 17814.66901 2545.057922 15269.61109 17814.66901 448141 448101 448141 448101 5.249924 0\.04 2707.391898 15755.96321 18463.35511 2707.391898 15755.96321 18463.35511 511800 511763 511800 511763 5.247974 0\.045 2938.631156 16421.48544 19360.11659 2938.631156 16421.48544 19360.11659 575744 575683 575744 575683 5.25208 0\.05 3354.394007 17314.71256 20669.10657 3354.394007 17314.71256 20669.10657 639705 639644 639705 639644 5.250088 0\.055 4301.373952 18705.94991 23007.32386 4301.373952 18705.94991 23007.32386 703661 703591 703661 703591 5.25145 0\.06 7323.253138 20918.07593 28241.32906 7323.253138 20918.07593 28241.32906 767327 767247 767327 767247 5.247669 0\.065 33369.62551 25298.39128 58668.01679 33369.62551 25298.39128 58668.01679 831131 831033 831131 831033 5.251095 0\.07 2658109.712 29815.79864 2687925.51 2658109.712 29815.79864 2687925.51 869634 869505 869634 869505 5.223994\ \ thank you!

KT

Krishna, Tushar

Fri, Oct 11, 2024 8:45 PM

Garnet doesn’t let more than one packet per VC .. so with single flit packets and a single VC the credit turnaround time will limit throughput ..

My guess is the paper you mentioned allows multiple packets per VC.. bcoz that’s what strictly wormhole is ..

Cheers,
Tushar

On Oct 11, 2024, at 4:35 PM, akavaler--- via gem5-users gem5-users@gem5.org wrote: