Packet VALID_ADDR being cleared when try to resend it !
P {margin-top:0;margin-bottom:0;} Hi all
What does it make a Packet's "VALID_ADDR" flag being cleared , even it was set before?
- I'm sending some packets from the cache to unblocking memory object that I created
- Inside this unblocking memory object, I crated a queue to store packets that fail to be sent
- After a while , this memory object receives a packet that failed to be sent , so this packet is blocked.
- Then, I'm inserting the packet to the queue of blocked requests.
- Inside the recvReqRetry I'm calling sendTimingReq(pkt).
- Eventually this packet is sent to CoherentXBar::recvTimingReq it fails there.
The following assertion fails:
gem5.debug: build/RISCV/mem/packet.hh:832: bool gem5::Packet::isSecure() const: Assertion `flags.isSet(VALID_ADDR)' failed.
Even If I tried to enable any packet debugging method (like calling pkt->print()) a segmentation fault arises!
I can find only that Packet's flags.clear() happens :
a. happens when constructing a new Packet from request, but even with that the "VALID_ADDR" is being set if it has a PAddr.
(There are two constructors)
b. Another case , is for a third constructor that takes another packet as an argument, but this constructor set the "VALID_ADDR" and "VALID_SIZE" to be the same as the packet being passed.
I can't find anywhere that flags.clear(VALID_ADDR) happens.
So, what can be the problem cause?
Thank you.
regards,
Abdelrhman
P {margin-top:0;margin-bottom:0;} After through debugging
It appears that the problem happens at the following call:
Inside " CoherentXBar::recvTimingReq":
success = memSidePorts[mem_side_port_id]->sendTimingReq(pkt);
This "sendTimingReq" appears to cause Pkt modifications.
This function actually will end up calling recvTimingReq at the memory controller.
I found an intersteing comment few lines before calling this function:
// store the original address as an address mapper could possibly
// modify the address upon a sendTimingRequest
const Addr addr(pkt->getAddr());
I still have no clue what is happening , but this converges the problem search if one can help.
Still, if I tried to "print" the packet (i.e. pkt->print()) anywhere on the path (or at least enable CoherentXBar debug flag) , it will cause a segmentation fault!
Thank you.
regards,
Abdelrhman
From: Abdlerhman Abotaleb <abotalea@mcmaster.ca>
Sent: Wednesday, July 13, 2022 11:52 AM
To: Balazs Gerofi via gem5-users <gem5-users@gem5.org>
Subject: [gem5-users] Packet VALID_ADDR being cleared when try to resend it !
Hi all
What does it make a Packet's "VALID_ADDR" flag being cleared , even it was set before?
- I'm sending some packets from the cache to unblocking memory object that I created
- Inside this unblocking memory object, I crated a queue to store packets that fail to be sent
- After a while , this memory object receives a packet that failed to be sent , so this packet is blocked.
- Then, I'm inserting the packet to the queue of blocked requests.
- Inside the recvReqRetry I'm calling sendTimingReq(pkt).
- Eventually this packet is sent to CoherentXBar::recvTimingReq it fails there.
The following assertion fails:
gem5.debug: build/RISCV/mem/packet.hh:832: bool gem5::Packet::isSecure() const: Assertion `flags.isSet(VALID_ADDR)' failed.
Even If I tried to enable any packet debugging method (like calling pkt->print()) a segmentation fault arises!
I can find only that Packet's flags.clear() happens :
a. happens when constructing a new Packet from request, but even with that the "VALID_ADDR" is being set if it has a PAddr.
(There are two constructors)
b. Another case , is for a third constructor that takes another packet as an argument, but this constructor set the "VALID_ADDR" and "VALID_SIZE" to be the same as the packet being passed.
I can't find anywhere that flags.clear(VALID_ADDR) happens.
So, what can be the problem cause?
Thank you.
regards,
Abdelrhman
On 7/13/2022 5:14 PM, Abdlerhman Abotaleb wrote:
After through debugging
It appears that the problem happens at the following call:
Inside "CoherentXBar::recvTimingReq":
success = memSidePorts[mem_side_port_id]->sendTimingReq(pkt);
This "sendTimingReq" appears to cause Pkt modifications.
This function actually will end up calling *recvTimingReq *at the memory controller.
I found an intersteing comment few lines before calling this function:
// store the original address as an address mapper could possibly
// modify the address upon a sendTimingRequest
const Addr addr(pkt->getAddr());
I still have no clue what is happening , but this converges the problem search if one can help.
Still, if I tried to "print" the packet (i.e. pkt->print()) anywhere on the path (or at least enable
CoherentXBar debug flag) , it will cause a segmentation fault!
I looked at that code, and while it indicates that the address can be changed
via mapping, I don't think that would lead to a problem, though of course I
could be wrong. What occurred to me was that, since you are queueing these
things for later processing, maybe the place that allocated them expected a
shorter lifetime, so part of the memory is getting freed and reused, which
could cause all kinds of oddities. Among other things, this could happen if
the Packet object is local to some function, rather than being allocated
explicitly with new and later disposed of with an explicit delete.
This thought also could be wrong, but might suggest a place to look.
While it can be tricky to trigger on just the right thing in gdb, if the
packet's address in memory, or the address being accessed, are repeatable, you
can set a conditional breakpoint at the place where the packet is created, or
other places along the path of its processing, and possibly find where things
go off the rails.
Regards - Eliot Moss
From: Abdlerhman Abotaleb abotalea@mcmaster.ca
Sent: Wednesday, July 13, 2022 11:52 AM
To: Balazs Gerofi via gem5-users gem5-users@gem5.org
Subject: [gem5-users] Packet VALID_ADDR being cleared when try to resend it !
Hi all
What does it make a Packet's "VALID_ADDR" flag being cleared , even it was set before?
- I'm sending some packets from the cache to unblocking memory object that I created
- Inside this unblocking memory object, I crated a queue to store packets that fail to be sent
- After a while , this memory object receives a packet that failed to be sent , so this packet is
blocked. - Then, I'm inserting the packet to the queue of blocked requests.
- Inside the recvReqRetry I'm calling sendTimingReq(pkt).
- Eventually this packet is sent to CoherentXBar::recvTimingReq it fails there.
The following assertion fails:
gem5.debug: build/RISCV/mem/packet.hh:832: bool gem5::Packet::isSecure() const:Assertion
`flags.isSet(VALID_ADDR)' failed.
Even If I tried to enable any packet debugging method (like calling pkt->print()) a segmentation
fault arises!
I can find only that Packet's flags.clear() happens :
a. happens when constructing a new Packet from request, but even with that the "VALID_ADDR" is
being set if it has a PAddr.
(There are two constructors)
b. Another case , is for a third constructor that takes another packet as an argument, but
this constructor set the "VALID_ADDR" and "VALID_SIZE" to be the same as the packet being passed.
I can't find anywhere that flags.clear(VALID_ADDR) happens.
So, what can be the problem cause?
Thank you.
regards,
Abdelrhman
gem5-users mailing list -- gem5-users@gem5.org
To unsubscribe send an email to gem5-users-leave@gem5.org
P {margin-top:0;margin-bottom:0;} Thank you, Eliot, for your reply. 🙏
I solved it, but still need to understand the cause.
This is the full story:
I defined an STL queue of pointers to Packets to store the packets for further resending.
std::queue<PacketPtr> pktQueue;
It is defined as a member of the memory object class.
And I'm debugging before and after this call
Inside " CoherentXBar::recvTimingReq":
success = memSidePorts[mem_side_port_id]->sendTimingReq(pkt);
Before the call, it appears that the Packet contains all correct fields (i.e. the address and flags etc.)
After the call it fails to assert the "VALID_ADDR".
I solved this problem as the following:
Inside the new memory simulation object, when inserting a new blocked packet:
Before:
blockedPacketQueue.push(pkt)
After:
PacketPtr pPkt= new Packet(pkt,false,true);
pktQueue.push(pPkt);
I still can't figure why the first way doesn't work !
From: Eliot Moss <moss@cs.umass.edu>
Sent: Wednesday, July 13, 2022 6:27 PM
To: The gem5 Users mailing list <gem5-users@gem5.org>
Subject: [gem5-users] Re: Packet VALID_ADDR being cleared when try to resend it !
On 7/13/2022 5:14 PM, Abdlerhman Abotaleb wrote:
> After through debugging
> It appears that the problem happens at the following call:
> Inside "CoherentXBar::recvTimingReq":
> success = memSidePorts[mem_side_port_id]->sendTimingReq(pkt);
> This "sendTimingReq" appears to cause Pkt modifications.
> This function actually will end up calling *recvTimingReq *at the memory controller.
>
> I found an intersteing comment few lines before calling this function:
> // store the original address as an address mapper could possibly
> // modify the address upon a sendTimingRequest
> const Addr addr(pkt->getAddr());
>
> I still have no clue what is happening , but this converges the problem search if one can help.
> Still, if I tried to "print" the packet (i.e. pkt->print()) anywhere on the path (or at least enable
> CoherentXBar debug flag) , it will cause a segmentation fault!
I looked at that code, and while it indicates that the address can be changed
via mapping, I don't think that would lead to a problem, though of course I
could be wrong. What occurred to me was that, since you are queueing these
things for later processing, maybe the place that allocated them expected a
shorter lifetime, so part of the memory is getting freed and reused, which
could cause all kinds of oddities. Among other things, this could happen if
the Packet object is local to some function, rather than being allocated
explicitly with new and later disposed of with an explicit delete.
This thought also could be wrong, but might suggest a place to look.
While it can be tricky to trigger on just the right thing in gdb, if the
packet's address in memory, or the address being accessed, are repeatable, you
can set a conditional breakpoint at the place where the packet is created, or
other places along the path of its processing, and possibly find where things
go off the rails.
Regards - Eliot Moss
> ----------------------------------------------------------------------------------------------------
> From: Abdlerhman Abotaleb <abotalea@mcmaster.ca>
> Sent: Wednesday, July 13, 2022 11:52 AM
> To: Balazs Gerofi via gem5-users <gem5-users@gem5.org>
> Subject: [gem5-users] Packet VALID_ADDR being cleared when try to resend it !
> Hi all
>
> What does it make a Packet's "VALID_ADDR" flag being cleared , even it was set before?
>
> 1. I'm sending some packets from the cache to unblocking memory object that I created
> 2. Inside this unblocking memory object, I crated a queue to store packets that fail to be sent
> 3. After a while , this memory object receives a packet that failed to be sent , so this packet is
> blocked.
> 4. Then, I'm inserting the packet to the queue of blocked requests.
> 5. Inside the recvReqRetry I'm calling sendTimingReq(pkt).
> 6. Eventually this packet is sent to CoherentXBar::recvTimingReq it fails there.
>
> The following assertion fails:
> gem5.debug: build/RISCV/mem/packet.hh:832: bool gem5::Packet::isSecure() const:Assertion
> `flags.isSet(VALID_ADDR)' failed.
> Even If I tried to enable any packet debugging method (like calling pkt->print()) a segmentation
> fault arises!
>
> I can find only that Packet's flags.clear() happens :
> a. happens when constructing a new Packet from request, but even with that the "VALID_ADDR" is
> being set if it has a PAddr.
> (There are two constructors)
> b. Another case , is for a third constructor that takes another packet as an argument, but
> this constructor set the "VALID_ADDR" and "VALID_SIZE" to be the same as the packet being passed.
>
> I can't find anywhere that flags.clear(VALID_ADDR) happens.
> So, what can be the problem cause?
>
> Thank you.
>
> regards,
> Abdelrhman
>
>
>
>
>
>
>
>
>
> _______________________________________________
> gem5-users mailing list -- gem5-users@gem5.org
> To unsubscribe send an email to gem5-users-leave@gem5.org
>
gem5-users mailing list -- gem5-users@gem5.org
To unsubscribe send an email to gem5-users-leave@gem5.org
On 7/14/2022 2:46 PM, Abdlerhman Abotaleb wrote:
Thank you, Eliot, for your reply. 🙏
I solved it, but still need to understand the cause. *
*
*This is the full story: *
I defined an STL queue of pointers to Packets to store the packets for further resending.
std::queue<PacketPtr> pktQueue;
It is defined as a member of the memory object class.
And I'm debugging before and after this call
Inside "CoherentXBar::recvTimingReq":
success = memSidePorts[mem_side_port_id]->sendTimingReq(pkt);
Before the call, it appears that the Packet contains all correct fields (i.e. the address and
flags etc.)
*After * the call it fails to assert the "VALID_ADDR".
I *solved *this problem as the following:
Inside the newmemory simulation object, when inserting a new blocked packet:
Before:
blockedPacketQueue.push(pkt)
After:
PacketPtr pPkt= new Packet(pkt,false,true);
pktQueue.push(pPkt);
I still can't figure why the first way doesn't work !
Well, I still wonder if the packet is getting freed
(or something) down in the sendTimingReq actions.
Among other things, I wonder if you are trying to save
it and it is getting responded to, which would turn
it into a response, etc.
Copying it like that is fixing the symptom, but I am
not sure it is fixing the real problem.
Regards - Eliot
P {margin-top:0;margin-bottom:0;} Thanks Eliot for the follow up.
The reposne is created independently using the memory controller.
and the problem arises inside recvTimingReq, so it happens even before complete sending teh request.
I don't free up the pkt pointer "pkt", I pop the queue element, after the re-try manages to send the packet.
and pop doesn't do any allocation free-up.
From: Eliot Moss <moss@cs.umass.edu>
Sent: Thursday, July 14, 2022 4:21 PM
To: The gem5 Users mailing list <gem5-users@gem5.org>
Subject: [gem5-users] Re: Packet VALID_ADDR being cleared when try to resend it !
On 7/14/2022 2:46 PM, Abdlerhman Abotaleb wrote:
> Thank you, Eliot, for your reply. 🙏
> I solved it, but still need to understand the cause. *
> *
> *This is the full story: *
> I defined an STL queue of pointers to Packets to store the packets for further resending.
> std::queue<PacketPtr> pktQueue;
> It is defined as a member of the memory object class.
>
> And I'm debugging before and after this call
> Inside "CoherentXBar::recvTimingReq":
> success = memSidePorts[mem_side_port_id]->sendTimingReq(pkt);
> Before the call, it appears that the Packet contains all correct fields (i.e. the address and
> flags etc.)
> *After * the call it fails to assert the "VALID_ADDR".
>
> I *solved *this problem as the following:
> Inside the newmemory simulation object, when inserting a new blocked packet:
> Before:
> blockedPacketQueue.push(pkt)
> After:
> PacketPtr pPkt= new Packet(pkt,false,true);
> pktQueue.push(pPkt);
>
> I still can't figure why the first way doesn't work !
Well, I still wonder if the packet is getting freed
(or something) down in the sendTimingReq actions.
Among other things, I wonder if you are trying to save
it and it is getting responded to, which would turn
it into a response, etc.
Copying it like that is fixing the symptom, but I am
not sure it is fixing the real problem.
Regards - Eliot
gem5-users mailing list -- gem5-users@gem5.org
To unsubscribe send an email to gem5-users-leave@gem5.org
On 7/14/2022 8:07 PM, Abdlerhman Abotaleb wrote:
Thanks Eliot for the follow up.
The reposne is created independently using the memory controller.
and the problem arises inside recvTimingReq, so it happens even before complete sending teh request.
I don't free up the pkt pointer "pkt", I pop the queue element, after the re-try manages to send the
packet.
and pop doesn't do any allocation free-up.
I think we're talking a little at cross purposes.
I was suggesting that some of the existing gem5
code was somehow freeing the packet, while you
had retained a pointer to it. So there's nothing
exactly wrong with your code in that scenario
except maybe having a wrong expectation about
the packet's lifetime.
EM
P {margin-top:0;margin-bottom:0;} But in such case, GEM5 code will fail without my memory object being added and this is not the case, it was working perfectly.
Because, omitting the added simulation object, the packet pointer "pkt" is being sent to the crossbar then the memory controller.
From: Eliot Moss <moss@cs.umass.edu>
Sent: Thursday, July 14, 2022 8:48 PM
To: The gem5 Users mailing list <gem5-users@gem5.org>
Subject: [gem5-users] Re: Packet VALID_ADDR being cleared when try to resend it !
On 7/14/2022 8:07 PM, Abdlerhman Abotaleb wrote:
> Thanks Eliot for the follow up.
> The reposne is created independently using the memory controller.
> and the problem arises inside recvTimingReq, so it happens even before complete sending teh request.
>
> I don't free up the pkt pointer "pkt", I pop the queue element, after the re-try manages to send the
> packet.
> and pop doesn't do any allocation free-up.
I think we're talking a little at cross purposes.
I was suggesting that some of the existing gem5
code was somehow freeing the packet, while you
had retained a pointer to it. So there's nothing
exactly wrong with your code in that scenario
except maybe having a wrong expectation about
the packet's lifetime.
EM
gem5-users mailing list -- gem5-users@gem5.org
To unsubscribe send an email to gem5-users-leave@gem5.org
On 7/15/2022 7:54 AM, Abdlerhman Abotaleb wrote:
But in such case, GEM5 code will fail without my memory object being added and this is not the case,
it was working perfectly.
Because, omitting the added simulation object, the packet pointer "pkt" is being sent to the
crossbar then the memory controller.
Let me try again to explain the scenario I was thinking of.
The timing packet originates from a component such as the cpu. In stages, as
timing delay time are reached, it is forwarded, component to component, to the
memory, then it is turn into a response, and sent back through the components,
generally in several steps, according to delays. Then it is destroyed.
Your change put a pointer to the packet into some private queue - but the
packet was also sent along its regular path. By the time you pull the
packet out of your queue and look at it, it has flowed through the whole
regular process and been deleted.
By making a copy, you of course avoid that - but the packet may be being
processed twice.
As I've said, I could be wrong about what is going on, but this is the
possible scenario I was trying to explain.
Regards - Eliot