Injecting Synthetic Traffic
Hi all,
I have been using a traffic injector to inject traffic into the mesh.
However, I noticed that when I increase the injection rate, at some point
it saturates and I cannot inject more traffic even though the mesh and
memory bandwidth is not saturated.
For this specific study, I am only injecting traffic from one of the cpus
and when the injection interval is less than 3ns, I start seeing lots of
retries.
I am using the CHI protocol and the mesh frequency is set to 3GHz.
On paper I should be able to inject one request every cycle, but if I go
below one request every 6 cycles (3ns) I cannot inject more and only get
retries. For instance:
system.cpu00.data_sequencer.response_ports0: Request ReadReq for address
0xdf43623b did not issue because BufferFull
Any thoughts?
Thanks,
Amin
Just a minor edit, the mesh frequency is set to 2GHz.
On Mon, Jul 18, 2022 at 10:41 AM Amin Jadidi amin.jadidi@gmail.com wrote:
Hi all,
I have been using a traffic injector to inject traffic into the mesh.
However, I noticed that when I increase the injection rate, at some point
it saturates and I cannot inject more traffic even though the mesh and
memory bandwidth is not saturated.
For this specific study, I am only injecting traffic from one of the cpus
and when the injection interval is less than 3ns, I start seeing lots of
retries.
I am using the CHI protocol and the mesh frequency is set to 3GHz.
On paper I should be able to inject one request every cycle, but if I go
below one request every 6 cycles (3ns) I cannot inject more and only get
retries. For instance:
system.cpu00.data_sequencer.response_ports0: Request ReadReq for address
0xdf43623b did not issue because BufferFull
Any thoughts?
Thanks,
Amin
Hi Amin,
You might need to increase the number of outstanding requests in the sequencer (system.cpu*.data_sequencer.max_outstanding_requests param) and the cache controllers attached to the sequencer (e.g.: system.cpu*.l1d.number_of_TBEs, if you have a L2, that need to be adjusted as well).
You can also check the cache allocation parameters (e.g.: system.cpu*.l1d.alloc_on_seq_acc) to make sure all requests are going into the mesh and not being handled by the private caches (if that's your intent). Also, if you want all request to go to main memory, you may want to disable cache allocation at the HNFs. Notice the HNFs also have a number_of_TBEs param that you may want to tweak.
Best,
Tiago
From: Amin Jadidi amin.jadidi@gmail.com
Sent: Monday, July 18, 2022 1:14 PM
To: gem5-users@gem5.org gem5-users@gem5.org
Subject: [gem5-users] Re: Injecting Synthetic Traffic
Just a minor edit, the mesh frequency is set to 2GHz.
On Mon, Jul 18, 2022 at 10:41 AM Amin Jadidi <amin.jadidi@gmail.commailto:amin.jadidi@gmail.com> wrote:
Hi all,
I have been using a traffic injector to inject traffic into the mesh.
However, I noticed that when I increase the injection rate, at some point it saturates and I cannot inject more traffic even though the mesh and memory bandwidth is not saturated.
For this specific study, I am only injecting traffic from one of the cpus and when the injection interval is less than 3ns, I start seeing lots of retries.
I am using the CHI protocol and the mesh frequency is set to 3GHz.
On paper I should be able to inject one request every cycle, but if I go below one request every 6 cycles (3ns) I cannot inject more and only get retries. For instance:
system.cpu00.data_sequencer.response_ports0: Request ReadReq for address 0xdf43623b did not issue because BufferFull
Any thoughts?
Thanks,
Amin
IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
Hi Tiago,
Thanks for the information. I was able to increase the injected traffic
from the cpus by tweaking those parameters.
However, I have not been able to increase the injected traffic from the
DMAs/RNIs.
Even though the mesh frequency is 2GHz, the injected traffic from a single
DMA does not go above 8GB/S.
Any thoughts?
Best,
Amin
On Mon, Jul 18, 2022 at 3:27 PM Tiago Muck Tiago.Muck@arm.com wrote:
Hi Amin,
You might need to increase the number of outstanding requests in the
sequencer (system.cpu*.data_sequencer.max_outstanding_requests param) and
the cache controllers attached to the sequencer (e.g.: system.cpu*.l1d.number_of_TBEs,
if you have a L2, that need to be adjusted as well).
You can also check the cache allocation parameters (e.g.:
system.cpu*.l1d.alloc_on_seq_acc) to make sure all requests are going
into the mesh and not being handled by the private caches (if that's your
intent). Also, if you want all request to go to main memory, you may want
to disable cache allocation at the HNFs. Notice the HNFs also have a
number_of_TBEs param that you may want to tweak.
Best,
Tiago
From: Amin Jadidi amin.jadidi@gmail.com
Sent: Monday, July 18, 2022 1:14 PM
To: gem5-users@gem5.org gem5-users@gem5.org
Subject: [gem5-users] Re: Injecting Synthetic Traffic
Just a minor edit, the mesh frequency is set to 2GHz.
On Mon, Jul 18, 2022 at 10:41 AM Amin Jadidi amin.jadidi@gmail.com
wrote:
Hi all,
I have been using a traffic injector to inject traffic into the mesh.
However, I noticed that when I increase the injection rate, at some point
it saturates and I cannot inject more traffic even though the mesh and
memory bandwidth is not saturated.
For this specific study, I am only injecting traffic from one of the cpus
and when the injection interval is less than 3ns, I start seeing lots of
retries.
I am using the CHI protocol and the mesh frequency is set to 3GHz.
On paper I should be able to inject one request every cycle, but if I go
below one request every 6 cycles (3ns) I cannot inject more and only get
retries. For instance:
system.cpu00.data_sequencer.response_ports0: Request ReadReq for address
0xdf43623b did not issue because BufferFull
Any thoughts?
Thanks,
Amin
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy the
information in any medium. Thank you.
gem5-users mailing list -- gem5-users@gem5.org
To unsubscribe send an email to gem5-users-leave@gem5.org
Update: I was able to fix this. It took me a while to figure out the
plumbing of the sequencer for the RNIs.
On Wed, Jul 20, 2022 at 10:49 PM Amin Jadidi amin.jadidi@gmail.com wrote:
Hi Tiago,
Thanks for the information. I was able to increase the injected traffic
from the cpus by tweaking those parameters.
However, I have not been able to increase the injected traffic from the
DMAs/RNIs.
Even though the mesh frequency is 2GHz, the injected traffic from a single
DMA does not go above 8GB/S.
Any thoughts?
Best,
Amin
On Mon, Jul 18, 2022 at 3:27 PM Tiago Muck Tiago.Muck@arm.com wrote:
Hi Amin,
You might need to increase the number of outstanding requests in the
sequencer (system.cpu*.data_sequencer.max_outstanding_requests param)
and the cache controllers attached to the sequencer (e.g.: system.cpu*.l1d.number_of_TBEs,
if you have a L2, that need to be adjusted as well).
You can also check the cache allocation parameters (e.g.:
system.cpu*.l1d.alloc_on_seq_acc) to make sure all requests are going
into the mesh and not being handled by the private caches (if that's your
intent). Also, if you want all request to go to main memory, you may want
to disable cache allocation at the HNFs. Notice the HNFs also have a
number_of_TBEs param that you may want to tweak.
Best,
Tiago
From: Amin Jadidi amin.jadidi@gmail.com
Sent: Monday, July 18, 2022 1:14 PM
To: gem5-users@gem5.org gem5-users@gem5.org
Subject: [gem5-users] Re: Injecting Synthetic Traffic
Just a minor edit, the mesh frequency is set to 2GHz.
On Mon, Jul 18, 2022 at 10:41 AM Amin Jadidi amin.jadidi@gmail.com
wrote:
Hi all,
I have been using a traffic injector to inject traffic into the mesh.
However, I noticed that when I increase the injection rate, at some point
it saturates and I cannot inject more traffic even though the mesh and
memory bandwidth is not saturated.
For this specific study, I am only injecting traffic from one of the cpus
and when the injection interval is less than 3ns, I start seeing lots of
retries.
I am using the CHI protocol and the mesh frequency is set to 3GHz.
On paper I should be able to inject one request every cycle, but if I go
below one request every 6 cycles (3ns) I cannot inject more and only get
retries. For instance:
system.cpu00.data_sequencer.response_ports0: Request ReadReq for address
0xdf43623b did not issue because BufferFull
Any thoughts?
Thanks,
Amin
IMPORTANT NOTICE: The contents of this email and any attachments are
confidential and may also be privileged. If you are not the intended
recipient, please notify the sender immediately and do not disclose the
contents to any other person, use it for any purpose, or store or copy the
information in any medium. Thank you.
gem5-users mailing list -- gem5-users@gem5.org
To unsubscribe send an email to gem5-users-leave@gem5.org