gem5-users@gem5.org

The gem5 Users mailing list

View all threads

Python 3.11 on stable release

DC
Derek Christ
Thu, May 11, 2023 2:22 PM

Hello,

currently I have trouble building the most recent stable release of gem5
on Arch Linux since they switched from Python 3.10 to Python 3.11.

When simply building gem5, it errors out at the configuration stage of
scons:

Checking Python version... no
Error: Can't find a working Python installation

I figured by simply upgrading the pybind11 dependency in the ext/
directory manually fixes this error.

However, now the configuration fails at a later point in time:

Checking for pkg-config package grpc++... (cached) yes
ERROR:fm_proj_ply:/home/derek/Git/gem5/build/X86/arch/arm/fastmodel/SConscript:220:
Invalid regular expression for rule 't_STRLIT'. global flag
s not at the start of the expression at position 13
SyntaxError: Can't build lexer

I did not found a way to fix this yet.

Also, creating a virtual environment with Python 3.10 did not work for me:
The first error is fixed even with the old pybind version, but the
second error still occurs.

Is there an easy fix for this?

Thanks!

Best
Derek

Hello, currently I have trouble building the most recent stable release of gem5 on Arch Linux since they switched from Python 3.10 to Python 3.11. When simply building gem5, it errors out at the configuration stage of scons: Checking Python version... no Error: Can't find a working Python installation I figured by simply upgrading the pybind11 dependency in the ext/ directory manually fixes this error. However, now the configuration fails at a later point in time: Checking for pkg-config package grpc++... (cached) yes ERROR:fm_proj_ply:/home/derek/Git/gem5/build/X86/arch/arm/fastmodel/SConscript:220: Invalid regular expression for rule 't_STRLIT'. global flag s not at the start of the expression at position 13 SyntaxError: Can't build lexer I did not found a way to fix this yet. Also, creating a virtual environment with Python 3.10 did not work for me: The first error is fixed even with the old pybind version, but the second error still occurs. Is there an easy fix for this? Thanks! Best Derek
RC
Richard Cooper
Thu, May 11, 2023 3:04 PM

Hi Derek,

We have fixed this problem on the develop branch, but have not backported it to the stable branch yet.
You should be able to cherry-pick this commit to fix the problem:
https://gem5-review.googlesource.com/c/public/gem5/+/70237

For a venv, the problem is that the python3-config utility picks up the system-wide Python include and library locations, not the venv’s ones.
I’ve worked around this in the past by using a customized version of python-config that detects the Python version for the venv and returns appropriate include and library locations. Such a tool wouldn’t be very portable between distros, so you’d have to write one for your own environment.

Best regards,

Richard.

From: Derek Christ via gem5-users gem5-users@gem5.org
Sent: Thursday, May 11, 2023 3:22 PM
To: gem5-users@gem5.org
Cc: Derek Christ christ.derek@gmail.com
Subject: [gem5-users] Python 3.11 on stable release

Hello,

currently I have trouble building the most recent stable release of gem5 on Arch Linux since they switched from Python 3.10 to Python 3.11.

When simply building gem5, it errors out at the configuration stage of scons:

Checking Python version... no
Error: Can't find a working Python installation

I figured by simply upgrading the pybind11 dependency in the ext/ directory manually fixes this error.

However, now the configuration fails at a later point in time:

Checking for pkg-config package grpc++... (cached) yes
ERROR:fm_proj_ply:/home/derek/Git/gem5/build/X86/arch/arm/fastmodel/SConscript:220: Invalid regular expression for rule 't_STRLIT'. global flag
s not at the start of the expression at position 13
SyntaxError: Can't build lexer

I did not found a way to fix this yet.

Also, creating a virtual environment with Python 3.10 did not work for me:
The first error is fixed even with the old pybind version, but the second error still occurs.

Is there an easy fix for this?

Thanks!

Best
Derek

IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.

Hi Derek, We have fixed this problem on the `develop` branch, but have not backported it to the stable branch yet. You should be able to cherry-pick this commit to fix the problem: https://gem5-review.googlesource.com/c/public/gem5/+/70237 For a venv, the problem is that the python3-config utility picks up the system-wide Python include and library locations, not the venv’s ones. I’ve worked around this in the past by using a customized version of python-config that detects the Python version for the venv and returns appropriate include and library locations. Such a tool wouldn’t be very portable between distros, so you’d have to write one for your own environment. Best regards, Richard. From: Derek Christ via gem5-users <gem5-users@gem5.org> Sent: Thursday, May 11, 2023 3:22 PM To: gem5-users@gem5.org Cc: Derek Christ <christ.derek@gmail.com> Subject: [gem5-users] Python 3.11 on stable release Hello, currently I have trouble building the most recent stable release of gem5 on Arch Linux since they switched from Python 3.10 to Python 3.11. When simply building gem5, it errors out at the configuration stage of scons: Checking Python version... no Error: Can't find a working Python installation I figured by simply upgrading the pybind11 dependency in the ext/ directory manually fixes this error. However, now the configuration fails at a later point in time: Checking for pkg-config package grpc++... (cached) yes ERROR:fm_proj_ply:/home/derek/Git/gem5/build/X86/arch/arm/fastmodel/SConscript:220: Invalid regular expression for rule 't_STRLIT'. global flag s not at the start of the expression at position 13 SyntaxError: Can't build lexer I did not found a way to fix this yet. Also, creating a virtual environment with Python 3.10 did not work for me: The first error is fixed even with the old pybind version, but the second error still occurs. Is there an easy fix for this? Thanks! Best Derek IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
DC
Derek Christ
Tue, May 16, 2023 2:25 PM

Hi Richard,

thanks for your answer!

With the patch cherry-picked, it starts to compile now. Unfortunately,
it still fails at a later point in time:

[SO Param] m5.objects.BaseMMU, BaseMMU-> X86/params/BaseMMU.hh
terminate called after throwing an instance of
'pybind11::error_already_set'
 what():  AttributeError: module 'inspect' has no attribute 'getargspec'

At:
 build/X86/python/m5/SimObject.py(478): decorate
 build/X86/python/m5/SimObject.py(515): cxxMethod
 build/X86/python/m5/SimObject.py(634): SimObject
 build/X86/python/m5/SimObject.py(611): <module>
 <string>(40): exec_module
 <frozen importlib._bootstrap>(705): _load_unlocked
 <frozen importlib._bootstrap>(1152): _find_and_load_unlocked
 <frozen importlib._bootstrap>(1178): _find_and_load
 build/X86/arch/generic/BaseInterrupts.py(27): <module>
 <string>(40): exec_module
 <frozen importlib._bootstrap>(705): _load_unlocked
 <frozen importlib._bootstrap>(1152): _find_and_load_unlocked
 <frozen importlib._bootstrap>(1178): _find_and_load
 <string>(1): <module>
 build/X86/python/m5/objects/init.py(29): <module>
 <string>(40): exec_module
 <frozen importlib._bootstrap>(705): _load_unlocked
 <frozen importlib._bootstrap>(1152): _find_and_load_unlocked
 <frozen importlib._bootstrap>(1178): _find_and_load
 <frozen importlib._bootstrap>(1206): _gcd_import
 <frozen importlib._bootstrap>(241): _call_with_frames_removed
 <frozen importlib._bootstrap>(1128): _find_and_load_unlocked
 <frozen importlib._bootstrap>(1178): _find_and_load
 <frozen importlib._bootstrap>(1206): _gcd_import
 /usr/lib/python3.11/importlib/init.py(126): import_module
 build_tools/sim_object_param_struct_hh.py(58): <module>

/tmp/tmpvc7mcwi_: line 1: 27766 Aborted                 (core dumped)
"build/X86/gem5py_m5" "build_tools/sim_object_param_struct_hh.py"
"m5.objects.BaseMMU" "build/X86/params/BaseMMU.hh"
scons: *** [build/X86/params/BaseMMU.hh] Error 134
scons: building terminated because of errors.

I think there might be another patch I would have to apply?

Best
Derek

Hi Richard, thanks for your answer! With the patch cherry-picked, it starts to compile now. Unfortunately, it still fails at a later point in time: [SO Param] m5.objects.BaseMMU, BaseMMU-> X86/params/BaseMMU.hh terminate called after throwing an instance of 'pybind11::error_already_set'  what():  AttributeError: module 'inspect' has no attribute 'getargspec' At:  build/X86/python/m5/SimObject.py(478): decorate  build/X86/python/m5/SimObject.py(515): cxxMethod  build/X86/python/m5/SimObject.py(634): SimObject  build/X86/python/m5/SimObject.py(611): <module>  <string>(40): exec_module  <frozen importlib._bootstrap>(705): _load_unlocked  <frozen importlib._bootstrap>(1152): _find_and_load_unlocked  <frozen importlib._bootstrap>(1178): _find_and_load  build/X86/arch/generic/BaseInterrupts.py(27): <module>  <string>(40): exec_module  <frozen importlib._bootstrap>(705): _load_unlocked  <frozen importlib._bootstrap>(1152): _find_and_load_unlocked  <frozen importlib._bootstrap>(1178): _find_and_load  <string>(1): <module>  build/X86/python/m5/objects/__init__.py(29): <module>  <string>(40): exec_module  <frozen importlib._bootstrap>(705): _load_unlocked  <frozen importlib._bootstrap>(1152): _find_and_load_unlocked  <frozen importlib._bootstrap>(1178): _find_and_load  <frozen importlib._bootstrap>(1206): _gcd_import  <frozen importlib._bootstrap>(241): _call_with_frames_removed  <frozen importlib._bootstrap>(1128): _find_and_load_unlocked  <frozen importlib._bootstrap>(1178): _find_and_load  <frozen importlib._bootstrap>(1206): _gcd_import  /usr/lib/python3.11/importlib/__init__.py(126): import_module  build_tools/sim_object_param_struct_hh.py(58): <module> /tmp/tmpvc7mcwi_: line 1: 27766 Aborted                 (core dumped) "build/X86/gem5py_m5" "build_tools/sim_object_param_struct_hh.py" "m5.objects.BaseMMU" "build/X86/params/BaseMMU.hh" scons: *** [build/X86/params/BaseMMU.hh] Error 134 scons: building terminated because of errors. I think there might be another patch I would have to apply? Best Derek
Z
zhangcongwu
Tue, May 16, 2023 2:58 PM

Hi Derek,

I have built it successfully with cherry-pick another two commits https://gem5-review.googlesource.com/c/public/gem5/+/68818?usp=search and https://gem5-review.googlesource.com/c/public/gem5/+/68817. You can try these two commits.

Best
Congwu

On May 16, 2023, at 22:25, Derek Christ via gem5-users gem5-users@gem5.org wrote:

Hi Richard,

thanks for your answer!

With the patch cherry-picked, it starts to compile now. Unfortunately, it still fails at a later point in time:

[SO Param] m5.objects.BaseMMU, BaseMMU -> X86/params/BaseMMU.hh
terminate called after throwing an instance of 'pybind11::error_already_set'
what():  AttributeError: module 'inspect' has no attribute 'getargspec'

At:
build/X86/python/m5/SimObject.py(478): decorate
build/X86/python/m5/SimObject.py(515): cxxMethod
build/X86/python/m5/SimObject.py(634): SimObject
build/X86/python/m5/SimObject.py(611): <module>
<string>(40): exec_module
<frozen importlib._bootstrap>(705): _load_unlocked
<frozen importlib._bootstrap>(1152): _find_and_load_unlocked
<frozen importlib._bootstrap>(1178): _find_and_load
build/X86/arch/generic/BaseInterrupts.py(27): <module>
<string>(40): exec_module
<frozen importlib._bootstrap>(705): _load_unlocked
<frozen importlib._bootstrap>(1152): _find_and_load_unlocked
<frozen importlib._bootstrap>(1178): _find_and_load
<string>(1): <module>
build/X86/python/m5/objects/init.py(29): <module>
<string>(40): exec_module
<frozen importlib._bootstrap>(705): _load_unlocked
<frozen importlib._bootstrap>(1152): _find_and_load_unlocked
<frozen importlib._bootstrap>(1178): _find_and_load
<frozen importlib._bootstrap>(1206): _gcd_import
<frozen importlib._bootstrap>(241): _call_with_frames_removed
<frozen importlib._bootstrap>(1128): _find_and_load_unlocked
<frozen importlib._bootstrap>(1178): _find_and_load
<frozen importlib._bootstrap>(1206): _gcd_import
/usr/lib/python3.11/importlib/init.py(126): import_module
build_tools/sim_object_param_struct_hh.py(58): <module>

/tmp/tmpvc7mcwi_: line 1: 27766 Aborted                (core dumped) "build/X86/gem5py_m5" "build_tools/sim_object_param_struct_hh.py" "m5.objects.BaseMMU" "build/X86/params/BaseMMU.hh"
scons: *** [build/X86/params/BaseMMU.hh] Error 134
scons: building terminated because of errors.

I think there might be another patch I would have to apply?

Best
Derek


gem5-users mailing list -- gem5-users@gem5.org
To unsubscribe send an email to gem5-users-leave@gem5.org

Hi Derek, I have built it successfully with cherry-pick another two commits https://gem5-review.googlesource.com/c/public/gem5/+/68818?usp=search and https://gem5-review.googlesource.com/c/public/gem5/+/68817. You can try these two commits. Best Congwu > On May 16, 2023, at 22:25, Derek Christ via gem5-users <gem5-users@gem5.org> wrote: > > Hi Richard, > > thanks for your answer! > > With the patch cherry-picked, it starts to compile now. Unfortunately, it still fails at a later point in time: > > [SO Param] m5.objects.BaseMMU, BaseMMU -> X86/params/BaseMMU.hh > terminate called after throwing an instance of 'pybind11::error_already_set' > what(): AttributeError: module 'inspect' has no attribute 'getargspec' > > At: > build/X86/python/m5/SimObject.py(478): decorate > build/X86/python/m5/SimObject.py(515): cxxMethod > build/X86/python/m5/SimObject.py(634): SimObject > build/X86/python/m5/SimObject.py(611): <module> > <string>(40): exec_module > <frozen importlib._bootstrap>(705): _load_unlocked > <frozen importlib._bootstrap>(1152): _find_and_load_unlocked > <frozen importlib._bootstrap>(1178): _find_and_load > build/X86/arch/generic/BaseInterrupts.py(27): <module> > <string>(40): exec_module > <frozen importlib._bootstrap>(705): _load_unlocked > <frozen importlib._bootstrap>(1152): _find_and_load_unlocked > <frozen importlib._bootstrap>(1178): _find_and_load > <string>(1): <module> > build/X86/python/m5/objects/__init__.py(29): <module> > <string>(40): exec_module > <frozen importlib._bootstrap>(705): _load_unlocked > <frozen importlib._bootstrap>(1152): _find_and_load_unlocked > <frozen importlib._bootstrap>(1178): _find_and_load > <frozen importlib._bootstrap>(1206): _gcd_import > <frozen importlib._bootstrap>(241): _call_with_frames_removed > <frozen importlib._bootstrap>(1128): _find_and_load_unlocked > <frozen importlib._bootstrap>(1178): _find_and_load > <frozen importlib._bootstrap>(1206): _gcd_import > /usr/lib/python3.11/importlib/__init__.py(126): import_module > build_tools/sim_object_param_struct_hh.py(58): <module> > > /tmp/tmpvc7mcwi_: line 1: 27766 Aborted (core dumped) "build/X86/gem5py_m5" "build_tools/sim_object_param_struct_hh.py" "m5.objects.BaseMMU" "build/X86/params/BaseMMU.hh" > scons: *** [build/X86/params/BaseMMU.hh] Error 134 > scons: building terminated because of errors. > > I think there might be another patch I would have to apply? > > Best > Derek > > _______________________________________________ > gem5-users mailing list -- gem5-users@gem5.org > To unsubscribe send an email to gem5-users-leave@gem5.org
DC
Derek Christ
Tue, May 16, 2023 3:26 PM

Hi Congwu,

with these two commits, it now works! Thank you both!

Best
Derek

Am 16.05.23 um 16:58 schrieb zhangcongwu via gem5-users:

Hi Derek,

I have built it successfully with cherry-pick another two commits
https://gem5-review.googlesource.com/c/public/gem5/+/68818?usp=search and
https://gem5-review.googlesource.com/c/public/gem5/+/68817. You can
try these two commits.

Best
Congwu

On May 16, 2023, at 22:25, Derek Christ via gem5-users
gem5-users@gem5.org wrote:

Hi Richard,

thanks for your answer!

With the patch cherry-picked, it starts to compile now.
Unfortunately, it still fails at a later point in time:

[SO Param] m5.objects.BaseMMU, BaseMMU-> X86/params/BaseMMU.hh
terminate called after throwing an instance of
'pybind11::error_already_set'
 what():  AttributeError: module 'inspect' has no attribute 'getargspec'

At:
 build/X86/python/m5/SimObject.py(478): decorate
 build/X86/python/m5/SimObject.py(515): cxxMethod
 build/X86/python/m5/SimObject.py(634): SimObject
 build/X86/python/m5/SimObject.py(611): <module>
 <string>(40): exec_module
 <frozen importlib._bootstrap>(705): _load_unlocked
 <frozen importlib._bootstrap>(1152): _find_and_load_unlocked
 <frozen importlib._bootstrap>(1178): _find_and_load
 build/X86/arch/generic/BaseInterrupts.py(27): <module>
 <string>(40): exec_module
 <frozen importlib._bootstrap>(705): _load_unlocked
 <frozen importlib._bootstrap>(1152): _find_and_load_unlocked
 <frozen importlib._bootstrap>(1178): _find_and_load
 <string>(1): <module>
 build/X86/python/m5/objects/init.py(29): <module>
 <string>(40): exec_module
 <frozen importlib._bootstrap>(705): _load_unlocked
 <frozen importlib._bootstrap>(1152): _find_and_load_unlocked
 <frozen importlib._bootstrap>(1178): _find_and_load
 <frozen importlib._bootstrap>(1206): _gcd_import
 <frozen importlib._bootstrap>(241): _call_with_frames_removed
 <frozen importlib._bootstrap>(1128): _find_and_load_unlocked
 <frozen importlib._bootstrap>(1178): _find_and_load
 <frozen importlib._bootstrap>(1206): _gcd_import
 /usr/lib/python3.11/importlib/init.py(126): import_module
 build_tools/sim_object_param_struct_hh.py(58): <module>

/tmp/tmpvc7mcwi_: line 1: 27766 Aborted                 (core dumped)
"build/X86/gem5py_m5" "build_tools/sim_object_param_struct_hh.py"
"m5.objects.BaseMMU" "build/X86/params/BaseMMU.hh"
scons: *** [build/X86/params/BaseMMU.hh] Error 134
scons: building terminated because of errors.

I think there might be another patch I would have to apply?

Best
Derek


gem5-users mailing list -- gem5-users@gem5.org
To unsubscribe send an email to gem5-users-leave@gem5.org


gem5-users mailing list --gem5-users@gem5.org
To unsubscribe send an email togem5-users-leave@gem5.org

Hi Congwu, with these two commits, it now works! Thank you both! Best Derek Am 16.05.23 um 16:58 schrieb zhangcongwu via gem5-users: > Hi Derek, > > I have built it successfully with cherry-pick another two commits > https://gem5-review.googlesource.com/c/public/gem5/+/68818?usp=search and > https://gem5-review.googlesource.com/c/public/gem5/+/68817. You can > try these two commits. > > Best > Congwu > >> On May 16, 2023, at 22:25, Derek Christ via gem5-users >> <gem5-users@gem5.org> wrote: >> >> Hi Richard, >> >> thanks for your answer! >> >> With the patch cherry-picked, it starts to compile now. >> Unfortunately, it still fails at a later point in time: >> >> [SO Param] m5.objects.BaseMMU, BaseMMU-> X86/params/BaseMMU.hh >> terminate called after throwing an instance of >> 'pybind11::error_already_set' >>  what():  AttributeError: module 'inspect' has no attribute 'getargspec' >> >> At: >>  build/X86/python/m5/SimObject.py(478): decorate >>  build/X86/python/m5/SimObject.py(515): cxxMethod >>  build/X86/python/m5/SimObject.py(634): SimObject >>  build/X86/python/m5/SimObject.py(611): <module> >>  <string>(40): exec_module >>  <frozen importlib._bootstrap>(705): _load_unlocked >>  <frozen importlib._bootstrap>(1152): _find_and_load_unlocked >>  <frozen importlib._bootstrap>(1178): _find_and_load >>  build/X86/arch/generic/BaseInterrupts.py(27): <module> >>  <string>(40): exec_module >>  <frozen importlib._bootstrap>(705): _load_unlocked >>  <frozen importlib._bootstrap>(1152): _find_and_load_unlocked >>  <frozen importlib._bootstrap>(1178): _find_and_load >>  <string>(1): <module> >>  build/X86/python/m5/objects/__init__.py(29): <module> >>  <string>(40): exec_module >>  <frozen importlib._bootstrap>(705): _load_unlocked >>  <frozen importlib._bootstrap>(1152): _find_and_load_unlocked >>  <frozen importlib._bootstrap>(1178): _find_and_load >>  <frozen importlib._bootstrap>(1206): _gcd_import >>  <frozen importlib._bootstrap>(241): _call_with_frames_removed >>  <frozen importlib._bootstrap>(1128): _find_and_load_unlocked >>  <frozen importlib._bootstrap>(1178): _find_and_load >>  <frozen importlib._bootstrap>(1206): _gcd_import >>  /usr/lib/python3.11/importlib/__init__.py(126): import_module >>  build_tools/sim_object_param_struct_hh.py(58): <module> >> >> /tmp/tmpvc7mcwi_: line 1: 27766 Aborted                 (core dumped) >> "build/X86/gem5py_m5" "build_tools/sim_object_param_struct_hh.py" >> "m5.objects.BaseMMU" "build/X86/params/BaseMMU.hh" >> scons: *** [build/X86/params/BaseMMU.hh] Error 134 >> scons: building terminated because of errors. >> >> I think there might be another patch I would have to apply? >> >> Best >> Derek >> >> _______________________________________________ >> gem5-users mailing list -- gem5-users@gem5.org >> To unsubscribe send an email to gem5-users-leave@gem5.org > > > _______________________________________________ > gem5-users mailing list --gem5-users@gem5.org > To unsubscribe send an email togem5-users-leave@gem5.org