gem5-users@gem5.org

The gem5 Users mailing list

View all threads

GCN3 Full System

PG
Pau Galindo Figuerola
Tue, Sep 19, 2023 11:02 AM

Hi,

Is it possible to use GCN3 model to run FS mode? I've done it with VEGA but
I'm interested in GCN3 architecture.

In case it is what would it require?

Thank you.

Regards,
Pau

Hi, Is it possible to use GCN3 model to run FS mode? I've done it with VEGA but I'm interested in GCN3 architecture. In case it is what would it require? Thank you. Regards, Pau
PM
Poremba, Matthew
Tue, Sep 19, 2023 2:29 PM

[Public]

Hi Pau,

It’s probably not possible without a lot of set up, but I have found that GPUs that are “not supported” sometimes still work anyway with certain compute stacks.  You would probably need to:

  • Find a BIOS for a GCN3 GPU or rip it from a real GPU – This is so the driver loads so at least this is required
  • The GPU must be a dGPU or you would need to write an IOMMU model in gem5 to use an APU
  • You may need to use an older software stack – These could be hard to find pre-packaged anymore and might require building from source
  • May need to rewrite applications with older software stack API – Dependent on previous step

This will certainly not be easy, even if possible.  Out of curiosity, which parts of GCN3 are you interested in that are different from Vega or Vega does not have?

-Matt

From: Pau Galindo Figuerola via gem5-users gem5-users@gem5.org
Sent: Tuesday, September 19, 2023 4:02 AM
To: gem5-users@gem5.org
Cc: Pau Galindo Figuerola pau.galindo.figuerola@estudiantat.upc.edu
Subject: [gem5-users] GCN3 Full System

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Hi,

Is it possible to use GCN3 model to run FS mode? I've done it with VEGA but I'm interested in GCN3 architecture.

In case it is what would it require?

Thank you.

Regards,
Pau

[Public] Hi Pau, It’s probably not possible without a lot of set up, but I have found that GPUs that are “not supported” sometimes still work anyway with certain compute stacks. You would probably need to: * Find a BIOS for a GCN3 GPU or rip it from a real GPU – This is so the driver loads so *at least* this is required * The GPU must be a dGPU or you would need to write an IOMMU model in gem5 to use an APU * You *may* need to use an older software stack – These could be hard to find pre-packaged anymore and might require building from source * May need to rewrite applications with older software stack API – Dependent on previous step This will certainly not be easy, even if possible. Out of curiosity, which parts of GCN3 are you interested in that are different from Vega or Vega does not have? -Matt From: Pau Galindo Figuerola via gem5-users <gem5-users@gem5.org> Sent: Tuesday, September 19, 2023 4:02 AM To: gem5-users@gem5.org Cc: Pau Galindo Figuerola <pau.galindo.figuerola@estudiantat.upc.edu> Subject: [gem5-users] GCN3 Full System Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding. Hi, Is it possible to use GCN3 model to run FS mode? I've done it with VEGA but I'm interested in GCN3 architecture. In case it is what would it require? Thank you. Regards, Pau
PG
Pau Galindo Figuerola
Fri, Apr 26, 2024 5:44 PM

Hi Matt,

I'm still working on it but I've decided to shift the project towards
modeling the AMD V1605B which has vega architecture.
I've the real AMD V1605B device and I'm wondering how could I rip the BIOS
from it. Do you have an idea on how to do it?

I build a custom image with a custom ROCm driver from Bruhnspace and I
suspect that it is now working because them are intendet to work for the
gtx902 gpu, which is the one V1605B has.
Do you think it has something to do with the fact that the BIOS provided in
the gem5 repositories is not exactly that one from the V1605B?

Thank you!

Best Regards,
Pau

Missatge de Poremba, Matthew Matthew.Poremba@amd.com del dia dt., 19 de
set. 2023 a les 16:29:

[Public]

Hi Pau,

It’s probably not possible without a lot of set up, but I have found that
GPUs that are “not supported” sometimes still work anyway with certain
compute stacks.  You would probably need to:

- Find a BIOS for a GCN3 GPU or rip it from a real GPU – This is so
the driver loads so **at least** this is required
- The GPU must be a dGPU or you would need to write an IOMMU model in
gem5 to use an APU
- You **may** need to use an older software stack – These could be
hard to find pre-packaged anymore and might require building from source
- May need to rewrite applications with older software stack API –
Dependent on previous step

This will certainly not be easy, even if possible.  Out of curiosity,
which parts of GCN3 are you interested in that are different from Vega or
Vega does not have?

-Matt

From: Pau Galindo Figuerola via gem5-users gem5-users@gem5.org
Sent: Tuesday, September 19, 2023 4:02 AM
To: gem5-users@gem5.org
Cc: Pau Galindo Figuerola pau.galindo.figuerola@estudiantat.upc.edu
Subject: [gem5-users] GCN3 Full System

Caution: This message originated from an External Source. Use proper
caution when opening attachments, clicking links, or responding.

Hi,

Is it possible to use GCN3 model to run FS mode? I've done it with VEGA
but I'm interested in GCN3 architecture.

In case it is what would it require?

Thank you.

Regards,

Pau

Hi Matt, I'm still working on it but I've decided to shift the project towards modeling the AMD V1605B which has vega architecture. I've the real AMD V1605B device and I'm wondering how could I rip the BIOS from it. Do you have an idea on how to do it? I build a custom image with a custom ROCm driver from Bruhnspace and I suspect that it is now working because them are intendet to work for the gtx902 gpu, which is the one V1605B has. Do you think it has something to do with the fact that the BIOS provided in the gem5 repositories is not exactly that one from the V1605B? Thank you! Best Regards, Pau Missatge de Poremba, Matthew <Matthew.Poremba@amd.com> del dia dt., 19 de set. 2023 a les 16:29: > [Public] > > Hi Pau, > > > > > > It’s probably not possible without a lot of set up, but I have found that > GPUs that are “not supported” sometimes still work anyway with certain > compute stacks. You would probably need to: > > - Find a BIOS for a GCN3 GPU or rip it from a real GPU – This is so > the driver loads so **at least** this is required > - The GPU must be a dGPU or you would need to write an IOMMU model in > gem5 to use an APU > - You **may** need to use an older software stack – These could be > hard to find pre-packaged anymore and might require building from source > - May need to rewrite applications with older software stack API – > Dependent on previous step > > > > This will certainly not be easy, even if possible. Out of curiosity, > which parts of GCN3 are you interested in that are different from Vega or > Vega does not have? > > > > > > -Matt > > > > *From:* Pau Galindo Figuerola via gem5-users <gem5-users@gem5.org> > *Sent:* Tuesday, September 19, 2023 4:02 AM > *To:* gem5-users@gem5.org > *Cc:* Pau Galindo Figuerola <pau.galindo.figuerola@estudiantat.upc.edu> > *Subject:* [gem5-users] GCN3 Full System > > > > *Caution:* This message originated from an External Source. Use proper > caution when opening attachments, clicking links, or responding. > > > > Hi, > > > > Is it possible to use GCN3 model to run FS mode? I've done it with VEGA > but I'm interested in GCN3 architecture. > > > > In case it is what would it require? > > > > Thank you. > > > > Regards, > > Pau >
PM
Poremba, Matthew
Fri, Apr 26, 2024 5:55 PM

[AMD Official Use Only - General]

Hi Pau,

You will likely need root access to rip the BIOS from a real GPU.  If you have that, I posted my notes below.  Keep in mind the PCI ids might be different:


You can find devices with ROM using find /sys/devices -name rom

For example, my MI210 is PCI device 83:00:

sudo bash -c 'echo 1 > /sys/devices/pci0000:80/0000:80:01.1/0000:81:00.0/0000:82:00.0/0000:83:00.0/rom'
sudo cat /sys/devices/pci0000:80/0000:80:01.1/0000:81:00.0/0000:82:00.0/0000:83:00.0/rom > test.rom
sudo bash -c 'echo 0 > /sys/devices/pci0000:80/0000:80:01.1/0000:81:00.0/0000:82:00.0/0000:83:00.0/rom'

In terms of the BIOS causing issues, I am not sure.  If you are running into issues associated with the gfx version, a hacky workaround is to use the following environment variable within gem5 to masquerade as another device, which might be worth a try. For example:

export HSA_OVERRIDE_GFX_VERSION="9.0.2"

-Matt

From: Pau Galindo Figuerola pau.galindo.figuerola@estudiantat.upc.edu
Sent: Friday, April 26, 2024 10:44 AM
To: Poremba, Matthew Matthew.Poremba@amd.com
Cc: The gem5 Users mailing list gem5-users@gem5.org
Subject: Re: [gem5-users] GCN3 Full System

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Hi Matt,

I'm still working on it but I've decided to shift the project towards modeling the AMD V1605B which has vega architecture.
I've the real AMD V1605B device and I'm wondering how could I rip the BIOS from it. Do you have an idea on how to do it?

I build a custom image with a custom ROCm driver from Bruhnspace and I suspect that it is now working because them are intendet to work for the gtx902 gpu, which is the one V1605B has.
Do you think it has something to do with the fact that the BIOS provided in the gem5 repositories is not exactly that one from the V1605B?

Thank you!

Best Regards,
Pau

Missatge de Poremba, Matthew <Matthew.Poremba@amd.commailto:Matthew.Poremba@amd.com> del dia dt., 19 de set. 2023 a les 16:29:

[Public]

Hi Pau,

It’s probably not possible without a lot of set up, but I have found that GPUs that are “not supported” sometimes still work anyway with certain compute stacks.  You would probably need to:

  • Find a BIOS for a GCN3 GPU or rip it from a real GPU – This is so the driver loads so at least this is required
  • The GPU must be a dGPU or you would need to write an IOMMU model in gem5 to use an APU
  • You may need to use an older software stack – These could be hard to find pre-packaged anymore and might require building from source
  • May need to rewrite applications with older software stack API – Dependent on previous step

This will certainly not be easy, even if possible.  Out of curiosity, which parts of GCN3 are you interested in that are different from Vega or Vega does not have?

-Matt

From: Pau Galindo Figuerola via gem5-users <gem5-users@gem5.orgmailto:gem5-users@gem5.org>
Sent: Tuesday, September 19, 2023 4:02 AM
To: gem5-users@gem5.orgmailto:gem5-users@gem5.org
Cc: Pau Galindo Figuerola <pau.galindo.figuerola@estudiantat.upc.edumailto:pau.galindo.figuerola@estudiantat.upc.edu>
Subject: [gem5-users] GCN3 Full System

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Hi,

Is it possible to use GCN3 model to run FS mode? I've done it with VEGA but I'm interested in GCN3 architecture.

In case it is what would it require?

Thank you.

Regards,
Pau

[AMD Official Use Only - General] Hi Pau, You will likely need root access to rip the BIOS from a real GPU. If you have that, I posted my notes below. Keep in mind the PCI ids might be different: “ You can find devices with ROM using `find /sys/devices -name rom` For example, my MI210 is PCI device 83:00: sudo bash -c 'echo 1 > /sys/devices/pci0000:80/0000:80:01.1/0000:81:00.0/0000:82:00.0/0000:83:00.0/rom' sudo cat /sys/devices/pci0000:80/0000:80:01.1/0000:81:00.0/0000:82:00.0/0000:83:00.0/rom > test.rom sudo bash -c 'echo 0 > /sys/devices/pci0000:80/0000:80:01.1/0000:81:00.0/0000:82:00.0/0000:83:00.0/rom' “ In terms of the BIOS causing issues, I am not sure. If you are running into issues associated with the gfx version, a hacky workaround is to use the following environment variable within gem5 to masquerade as another device, which might be worth a try. For example: export HSA_OVERRIDE_GFX_VERSION="9.0.2" -Matt From: Pau Galindo Figuerola <pau.galindo.figuerola@estudiantat.upc.edu> Sent: Friday, April 26, 2024 10:44 AM To: Poremba, Matthew <Matthew.Poremba@amd.com> Cc: The gem5 Users mailing list <gem5-users@gem5.org> Subject: Re: [gem5-users] GCN3 Full System Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding. Hi Matt, I'm still working on it but I've decided to shift the project towards modeling the AMD V1605B which has vega architecture. I've the real AMD V1605B device and I'm wondering how could I rip the BIOS from it. Do you have an idea on how to do it? I build a custom image with a custom ROCm driver from Bruhnspace and I suspect that it is now working because them are intendet to work for the gtx902 gpu, which is the one V1605B has. Do you think it has something to do with the fact that the BIOS provided in the gem5 repositories is not exactly that one from the V1605B? Thank you! Best Regards, Pau Missatge de Poremba, Matthew <Matthew.Poremba@amd.com<mailto:Matthew.Poremba@amd.com>> del dia dt., 19 de set. 2023 a les 16:29: [Public] Hi Pau, It’s probably not possible without a lot of set up, but I have found that GPUs that are “not supported” sometimes still work anyway with certain compute stacks. You would probably need to: * Find a BIOS for a GCN3 GPU or rip it from a real GPU – This is so the driver loads so *at least* this is required * The GPU must be a dGPU or you would need to write an IOMMU model in gem5 to use an APU * You *may* need to use an older software stack – These could be hard to find pre-packaged anymore and might require building from source * May need to rewrite applications with older software stack API – Dependent on previous step This will certainly not be easy, even if possible. Out of curiosity, which parts of GCN3 are you interested in that are different from Vega or Vega does not have? -Matt From: Pau Galindo Figuerola via gem5-users <gem5-users@gem5.org<mailto:gem5-users@gem5.org>> Sent: Tuesday, September 19, 2023 4:02 AM To: gem5-users@gem5.org<mailto:gem5-users@gem5.org> Cc: Pau Galindo Figuerola <pau.galindo.figuerola@estudiantat.upc.edu<mailto:pau.galindo.figuerola@estudiantat.upc.edu>> Subject: [gem5-users] GCN3 Full System Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding. Hi, Is it possible to use GCN3 model to run FS mode? I've done it with VEGA but I'm interested in GCN3 architecture. In case it is what would it require? Thank you. Regards, Pau
PG
Pau Galindo Figuerola
Mon, Apr 29, 2024 7:21 PM

Hi Matt,

I cannot find it in the pci devices because the v1605b is an APU that has
the GPU embedded. Any idea on where it could be?

By the way *export HSA_OVERRIDE_GFX_VERSION="9.0.2" *command don't seem to
work. The rocminfo and *clinfo *still detect the GPU as gfx900.
I have tried to modify the gpufs config to make the gfxVersion::gfx902
available from arguments and added the casuistries it to the
/src/dev/amdgpu/amdgpu_device.hh and */src/dev/amdgpu/amdgpu_device.cc *which
already had some code taking into account the gfx902 version.

Thank you!

Best Regards,
Pau

Missatge de Poremba, Matthew Matthew.Poremba@amd.com del dia dv., 26
d’abr. 2024 a les 19:55:

[AMD Official Use Only - General]

Hi Pau,

You will likely need root access to rip the BIOS from a real GPU.  If you
have that, I posted my notes below.  Keep in mind the PCI ids might be
different:

You can find devices with ROM using find /sys/devices -name rom

For example, my MI210 is PCI device 83:00:

sudo bash -c 'echo 1 >
/sys/devices/pci0000:80/0000:80:01.1/0000:81:00.0/0000:82:00.0/0000:83:00.0/rom'

sudo cat
/sys/devices/pci0000:80/0000:80:01.1/0000:81:00.0/0000:82:00.0/0000:83:00.0/rom

test.rom

sudo bash -c 'echo 0 >
/sys/devices/pci0000:80/0000:80:01.1/0000:81:00.0/0000:82:00.0/0000:83:00.0/rom'

In terms of the BIOS causing issues, I am not sure.  If you are running
into issues associated with the gfx version, a hacky workaround is to use
the following environment variable within gem5 to masquerade as another
device, which might be worth a try. For example:

export HSA_OVERRIDE_GFX_VERSION="9.0.2"

-Matt

From: Pau Galindo Figuerola pau.galindo.figuerola@estudiantat.upc.edu
Sent: Friday, April 26, 2024 10:44 AM
To: Poremba, Matthew Matthew.Poremba@amd.com
Cc: The gem5 Users mailing list gem5-users@gem5.org
Subject: Re: [gem5-users] GCN3 Full System

Caution: This message originated from an External Source. Use proper
caution when opening attachments, clicking links, or responding.

Hi Matt,

I'm still working on it but I've decided to shift the project towards
modeling the AMD V1605B which has vega architecture.

I've the real AMD V1605B device and I'm wondering how could I rip the BIOS
from it. Do you have an idea on how to do it?

I build a custom image with a custom ROCm driver from Bruhnspace and I
suspect that it is now working because them are intendet to work for the
gtx902 gpu, which is the one V1605B has.

Do you think it has something to do with the fact that the BIOS provided
in the gem5 repositories is not exactly that one from the V1605B?

Thank you!

Best Regards,

Pau

Missatge de Poremba, Matthew Matthew.Poremba@amd.com del dia dt., 19 de
set. 2023 a les 16:29:

[Public]

Hi Pau,

It’s probably not possible without a lot of set up, but I have found that
GPUs that are “not supported” sometimes still work anyway with certain
compute stacks.  You would probably need to:

- Find a BIOS for a GCN3 GPU or rip it from a real GPU – This is so
the driver loads so **at least** this is required
- The GPU must be a dGPU or you would need to write an IOMMU model in
gem5 to use an APU
- You **may** need to use an older software stack – These could be
hard to find pre-packaged anymore and might require building from source
- May need to rewrite applications with older software stack API –
Dependent on previous step

This will certainly not be easy, even if possible.  Out of curiosity,
which parts of GCN3 are you interested in that are different from Vega or
Vega does not have?

-Matt

From: Pau Galindo Figuerola via gem5-users gem5-users@gem5.org
Sent: Tuesday, September 19, 2023 4:02 AM
To: gem5-users@gem5.org
Cc: Pau Galindo Figuerola pau.galindo.figuerola@estudiantat.upc.edu
Subject: [gem5-users] GCN3 Full System

Caution: This message originated from an External Source. Use proper
caution when opening attachments, clicking links, or responding.

Hi,

Is it possible to use GCN3 model to run FS mode? I've done it with VEGA
but I'm interested in GCN3 architecture.

In case it is what would it require?

Thank you.

Regards,

Pau

Hi Matt, I cannot find it in the pci devices because the v1605b is an APU that has the GPU embedded. Any idea on where it could be? By the way *export HSA_OVERRIDE_GFX_VERSION="9.0.2" *command don't seem to work. The *rocminfo* and *clinfo *still detect the GPU as gfx900. I have tried to modify the gpufs config to make the gfxVersion::gfx902 available from arguments and added the casuistries it to the */src/dev/amdgpu/amdgpu_device.hh* and */src/dev/amdgpu/amdgpu_device.cc *which already had some code taking into account the gfx902 version. Thank you! Best Regards, Pau Missatge de Poremba, Matthew <Matthew.Poremba@amd.com> del dia dv., 26 d’abr. 2024 a les 19:55: > [AMD Official Use Only - General] > > Hi Pau, > > > > > > You will likely need root access to rip the BIOS from a real GPU. If you > have that, I posted my notes below. Keep in mind the PCI ids might be > different: > > > > “ > > You can find devices with ROM using `find /sys/devices -name rom` > > > > For example, my MI210 is PCI device 83:00: > > > > sudo bash -c 'echo 1 > > /sys/devices/pci0000:80/0000:80:01.1/0000:81:00.0/0000:82:00.0/0000:83:00.0/rom' > > sudo cat > /sys/devices/pci0000:80/0000:80:01.1/0000:81:00.0/0000:82:00.0/0000:83:00.0/rom > > test.rom > > sudo bash -c 'echo 0 > > /sys/devices/pci0000:80/0000:80:01.1/0000:81:00.0/0000:82:00.0/0000:83:00.0/rom' > > “ > > > > > > In terms of the BIOS causing issues, I am not sure. If you are running > into issues associated with the gfx version, a hacky workaround is to use > the following environment variable within gem5 to masquerade as another > device, which might be worth a try. For example: > > > > export HSA_OVERRIDE_GFX_VERSION="9.0.2" > > > > > > > > -Matt > > > > > > *From:* Pau Galindo Figuerola <pau.galindo.figuerola@estudiantat.upc.edu> > *Sent:* Friday, April 26, 2024 10:44 AM > *To:* Poremba, Matthew <Matthew.Poremba@amd.com> > *Cc:* The gem5 Users mailing list <gem5-users@gem5.org> > *Subject:* Re: [gem5-users] GCN3 Full System > > > > *Caution:* This message originated from an External Source. Use proper > caution when opening attachments, clicking links, or responding. > > > > Hi Matt, > > > > I'm still working on it but I've decided to shift the project towards > modeling the AMD V1605B which has vega architecture. > > I've the real AMD V1605B device and I'm wondering how could I rip the BIOS > from it. Do you have an idea on how to do it? > > > > I build a custom image with a custom ROCm driver from Bruhnspace and I > suspect that it is now working because them are intendet to work for the > gtx902 gpu, which is the one V1605B has. > > Do you think it has something to do with the fact that the BIOS provided > in the gem5 repositories is not exactly that one from the V1605B? > > > > Thank you! > > > > Best Regards, > > Pau > > > > Missatge de Poremba, Matthew <Matthew.Poremba@amd.com> del dia dt., 19 de > set. 2023 a les 16:29: > > [Public] > > > > Hi Pau, > > > > > > It’s probably not possible without a lot of set up, but I have found that > GPUs that are “not supported” sometimes still work anyway with certain > compute stacks. You would probably need to: > > - Find a BIOS for a GCN3 GPU or rip it from a real GPU – This is so > the driver loads so **at least** this is required > - The GPU must be a dGPU or you would need to write an IOMMU model in > gem5 to use an APU > - You **may** need to use an older software stack – These could be > hard to find pre-packaged anymore and might require building from source > - May need to rewrite applications with older software stack API – > Dependent on previous step > > > > This will certainly not be easy, even if possible. Out of curiosity, > which parts of GCN3 are you interested in that are different from Vega or > Vega does not have? > > > > > > -Matt > > > > *From:* Pau Galindo Figuerola via gem5-users <gem5-users@gem5.org> > *Sent:* Tuesday, September 19, 2023 4:02 AM > *To:* gem5-users@gem5.org > *Cc:* Pau Galindo Figuerola <pau.galindo.figuerola@estudiantat.upc.edu> > *Subject:* [gem5-users] GCN3 Full System > > > > *Caution:* This message originated from an External Source. Use proper > caution when opening attachments, clicking links, or responding. > > > > Hi, > > > > Is it possible to use GCN3 model to run FS mode? I've done it with VEGA > but I'm interested in GCN3 architecture. > > > > In case it is what would it require? > > > > Thank you. > > > > Regards, > > Pau > >
PM
Poremba, Matthew
Mon, Apr 29, 2024 10:24 PM

[AMD Official Use Only - General]

Hi Pau,

For an APU, I think you can get the bios from the vga rom region in memory.  There is a util file for this in gem5:  util/amdgpu/fs_files/dump_gpu_rom.sh .  However, if you have another GPU such as integrated graphics on the motherboard, it might get that bios instead.

-Matt

From: Pau Galindo Figuerola pau.galindo.figuerola@estudiantat.upc.edu
Sent: Monday, April 29, 2024 12:21 PM
To: Poremba, Matthew Matthew.Poremba@amd.com
Cc: The gem5 Users mailing list gem5-users@gem5.org
Subject: Re: [gem5-users] GCN3 Full System

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Hi Matt,

I cannot find it in the pci devices because the v1605b is an APU that has the GPU embedded. Any idea on where it could be?

By the way export HSA_OVERRIDE_GFX_VERSION="9.0.2" command don't seem to work. The rocminfo and clinfo still detect the GPU as gfx900.
I have tried to modify the gpufs config to make the gfxVersion::gfx902 available from arguments and added the casuistries it to the /src/dev/amdgpu/amdgpu_device.hh and /src/dev/amdgpu/amdgpu_device.cc which already had some code taking into account the gfx902 version.

Thank you!

Best Regards,
Pau

Missatge de Poremba, Matthew <Matthew.Poremba@amd.commailto:Matthew.Poremba@amd.com> del dia dv., 26 d’abr. 2024 a les 19:55:

[AMD Official Use Only - General]

Hi Pau,

You will likely need root access to rip the BIOS from a real GPU.  If you have that, I posted my notes below.  Keep in mind the PCI ids might be different:


You can find devices with ROM using find /sys/devices -name rom

For example, my MI210 is PCI device 83:00:

sudo bash -c 'echo 1 > /sys/devices/pci0000:80/0000:80:01.1/0000:81:00.0/0000:82:00.0/0000:83:00.0/rom'
sudo cat /sys/devices/pci0000:80/0000:80:01.1/0000:81:00.0/0000:82:00.0/0000:83:00.0/rom > test.rom
sudo bash -c 'echo 0 > /sys/devices/pci0000:80/0000:80:01.1/0000:81:00.0/0000:82:00.0/0000:83:00.0/rom'

In terms of the BIOS causing issues, I am not sure.  If you are running into issues associated with the gfx version, a hacky workaround is to use the following environment variable within gem5 to masquerade as another device, which might be worth a try. For example:

export HSA_OVERRIDE_GFX_VERSION="9.0.2"

-Matt

From: Pau Galindo Figuerola <pau.galindo.figuerola@estudiantat.upc.edumailto:pau.galindo.figuerola@estudiantat.upc.edu>
Sent: Friday, April 26, 2024 10:44 AM
To: Poremba, Matthew <Matthew.Poremba@amd.commailto:Matthew.Poremba@amd.com>
Cc: The gem5 Users mailing list <gem5-users@gem5.orgmailto:gem5-users@gem5.org>
Subject: Re: [gem5-users] GCN3 Full System

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Hi Matt,

I'm still working on it but I've decided to shift the project towards modeling the AMD V1605B which has vega architecture.
I've the real AMD V1605B device and I'm wondering how could I rip the BIOS from it. Do you have an idea on how to do it?

I build a custom image with a custom ROCm driver from Bruhnspace and I suspect that it is now working because them are intendet to work for the gtx902 gpu, which is the one V1605B has.
Do you think it has something to do with the fact that the BIOS provided in the gem5 repositories is not exactly that one from the V1605B?

Thank you!

Best Regards,
Pau

Missatge de Poremba, Matthew <Matthew.Poremba@amd.commailto:Matthew.Poremba@amd.com> del dia dt., 19 de set. 2023 a les 16:29:

[Public]

Hi Pau,

It’s probably not possible without a lot of set up, but I have found that GPUs that are “not supported” sometimes still work anyway with certain compute stacks.  You would probably need to:

  • Find a BIOS for a GCN3 GPU or rip it from a real GPU – This is so the driver loads so at least this is required
  • The GPU must be a dGPU or you would need to write an IOMMU model in gem5 to use an APU
  • You may need to use an older software stack – These could be hard to find pre-packaged anymore and might require building from source
  • May need to rewrite applications with older software stack API – Dependent on previous step

This will certainly not be easy, even if possible.  Out of curiosity, which parts of GCN3 are you interested in that are different from Vega or Vega does not have?

-Matt

From: Pau Galindo Figuerola via gem5-users <gem5-users@gem5.orgmailto:gem5-users@gem5.org>
Sent: Tuesday, September 19, 2023 4:02 AM
To: gem5-users@gem5.orgmailto:gem5-users@gem5.org
Cc: Pau Galindo Figuerola <pau.galindo.figuerola@estudiantat.upc.edumailto:pau.galindo.figuerola@estudiantat.upc.edu>
Subject: [gem5-users] GCN3 Full System

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Hi,

Is it possible to use GCN3 model to run FS mode? I've done it with VEGA but I'm interested in GCN3 architecture.

In case it is what would it require?

Thank you.

Regards,
Pau

[AMD Official Use Only - General] Hi Pau, For an APU, I think you can get the bios from the vga rom region in memory. There is a util file for this in gem5: util/amdgpu/fs_files/dump_gpu_rom.sh . However, if you have another GPU such as integrated graphics on the motherboard, it might get that bios instead. -Matt From: Pau Galindo Figuerola <pau.galindo.figuerola@estudiantat.upc.edu> Sent: Monday, April 29, 2024 12:21 PM To: Poremba, Matthew <Matthew.Poremba@amd.com> Cc: The gem5 Users mailing list <gem5-users@gem5.org> Subject: Re: [gem5-users] GCN3 Full System Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding. Hi Matt, I cannot find it in the pci devices because the v1605b is an APU that has the GPU embedded. Any idea on where it could be? By the way export HSA_OVERRIDE_GFX_VERSION="9.0.2" command don't seem to work. The rocminfo and clinfo still detect the GPU as gfx900. I have tried to modify the gpufs config to make the gfxVersion::gfx902 available from arguments and added the casuistries it to the /src/dev/amdgpu/amdgpu_device.hh and /src/dev/amdgpu/amdgpu_device.cc which already had some code taking into account the gfx902 version. Thank you! Best Regards, Pau Missatge de Poremba, Matthew <Matthew.Poremba@amd.com<mailto:Matthew.Poremba@amd.com>> del dia dv., 26 d’abr. 2024 a les 19:55: [AMD Official Use Only - General] Hi Pau, You will likely need root access to rip the BIOS from a real GPU. If you have that, I posted my notes below. Keep in mind the PCI ids might be different: “ You can find devices with ROM using `find /sys/devices -name rom` For example, my MI210 is PCI device 83:00: sudo bash -c 'echo 1 > /sys/devices/pci0000:80/0000:80:01.1/0000:81:00.0/0000:82:00.0/0000:83:00.0/rom' sudo cat /sys/devices/pci0000:80/0000:80:01.1/0000:81:00.0/0000:82:00.0/0000:83:00.0/rom > test.rom sudo bash -c 'echo 0 > /sys/devices/pci0000:80/0000:80:01.1/0000:81:00.0/0000:82:00.0/0000:83:00.0/rom' “ In terms of the BIOS causing issues, I am not sure. If you are running into issues associated with the gfx version, a hacky workaround is to use the following environment variable within gem5 to masquerade as another device, which might be worth a try. For example: export HSA_OVERRIDE_GFX_VERSION="9.0.2" -Matt From: Pau Galindo Figuerola <pau.galindo.figuerola@estudiantat.upc.edu<mailto:pau.galindo.figuerola@estudiantat.upc.edu>> Sent: Friday, April 26, 2024 10:44 AM To: Poremba, Matthew <Matthew.Poremba@amd.com<mailto:Matthew.Poremba@amd.com>> Cc: The gem5 Users mailing list <gem5-users@gem5.org<mailto:gem5-users@gem5.org>> Subject: Re: [gem5-users] GCN3 Full System Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding. Hi Matt, I'm still working on it but I've decided to shift the project towards modeling the AMD V1605B which has vega architecture. I've the real AMD V1605B device and I'm wondering how could I rip the BIOS from it. Do you have an idea on how to do it? I build a custom image with a custom ROCm driver from Bruhnspace and I suspect that it is now working because them are intendet to work for the gtx902 gpu, which is the one V1605B has. Do you think it has something to do with the fact that the BIOS provided in the gem5 repositories is not exactly that one from the V1605B? Thank you! Best Regards, Pau Missatge de Poremba, Matthew <Matthew.Poremba@amd.com<mailto:Matthew.Poremba@amd.com>> del dia dt., 19 de set. 2023 a les 16:29: [Public] Hi Pau, It’s probably not possible without a lot of set up, but I have found that GPUs that are “not supported” sometimes still work anyway with certain compute stacks. You would probably need to: * Find a BIOS for a GCN3 GPU or rip it from a real GPU – This is so the driver loads so *at least* this is required * The GPU must be a dGPU or you would need to write an IOMMU model in gem5 to use an APU * You *may* need to use an older software stack – These could be hard to find pre-packaged anymore and might require building from source * May need to rewrite applications with older software stack API – Dependent on previous step This will certainly not be easy, even if possible. Out of curiosity, which parts of GCN3 are you interested in that are different from Vega or Vega does not have? -Matt From: Pau Galindo Figuerola via gem5-users <gem5-users@gem5.org<mailto:gem5-users@gem5.org>> Sent: Tuesday, September 19, 2023 4:02 AM To: gem5-users@gem5.org<mailto:gem5-users@gem5.org> Cc: Pau Galindo Figuerola <pau.galindo.figuerola@estudiantat.upc.edu<mailto:pau.galindo.figuerola@estudiantat.upc.edu>> Subject: [gem5-users] GCN3 Full System Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding. Hi, Is it possible to use GCN3 model to run FS mode? I've done it with VEGA but I'm interested in GCN3 architecture. In case it is what would it require? Thank you. Regards, Pau