* Option to build ldpc_cuda was remaining in the build script
* Build script option to build ldpc_cuda was still used in CI
* Mentions of ldpc_cuda were remaining in the documentation
Note: In the documentation, the example of ldpctest usage that was
loading ldpc_cuda was replaced by an example in which it is loading
ldpc_orig even though this is pointless as it makes ldpctest compare
ldpc_orig with ldpc_orig. On the other hand, it is the only library
other than the default ldpc that uses the former interface.
CI: Add FHI7.2 phytest pipeline
This MR consists of multiple changes related to FHI7.2 setups in CI.
- remove unused fhi72 configurations in CI
- introduce dpdk-init service in fhi72 deployments to setup sriov on the machine
- update phytest fhi72 configurations for running on bulbul
- add phytest fhi72 to RAN-SA-FHI72-CN5G pipeline
- Introduce dpdk-init service for SR-IOV initialization within the container
- Execute script to enable low-latency RT performance (restored after tests)
When running phytest with FHI7.2 in Docker, two services are deployed:
dpdk-init and oai-gnb. Previously, AnalyzeRTStatsObject selected the
first deployed service for RT stats analysis.
Allow to explicitly select the service used for RT stats analysis instead
of relying on the first deployed service.
This extends the functionality provided by record in that it allows
collection of data into a Clickhouse database, allowing aggregation of
data from gNB and UE for functions like channel estimation for PUSCH
transmissions that did not decode successfully at the gNB.
It is intended to be used in conjunction with Aerial Data Lake but is
not limited to it. Data access is simpler than in !3462, but could be
improved using the metadata approach from !3462.
The integration uses logfmt for fast logging, and a clickhouse DB
client, both integrated using CPM.
This change set also adds a docker-compose file for a UE on 100MHz, to
be used with Aerial configurations, and builds the tracer for Aerial and
nrUE docker files.
CI: rework log analysis to simplify and allow new use cases
The CI has some log analysis, but it is unmaintainable (huge functions
that mix eNB/gNB or lteUE/nrUE, are not tested, most of it is not used
or useful...) or limited to specific use cases (the timing-phytest time
analysis is hardcoded to source builds instead of also handling analysis
when running from docker images). This MR reimplements both functions.
For the log analysis, the CI team decided to throw away the old
analysis, and reimplement a mechanism that checks the couple of things
we care for.
- Allow to analyze phy-test timing analysis from docker files, and add a
unit test
- Rework existing log analysis to remove useless checks, and make it
more composable
* The undeployment steps (Undeploy_Object(), Terminate_eNB()) take an
<analysis> XML object which then either has a <services> subelement
with check that are whitespace delimited (similar to how other xml
task steps have <services>). For checks that contain whitespace,
they can use <service> (singular) that encodes a single check, such
that e.g., arguments could include spaces.
* Each check is of form service=check[=arguments], where service is
the docker services, check is a checking function (implemented as a
class with run(file, options) function inside cls_loganalysis.py,
and options arguments after the second = that is to be read by run()
(basically, options will contain arguments that can be freeform)
* these definitions can occur multiple times, e.g., it could be
serviceA=Check1 and serviceA=Check2=Args and it will execute both
Check1 and Check2
* if a service is declared, it will execute alse the Default check
which currently checks for an assertion
* Implemented (and inspired from the existing CI analysis) are:
- Default check to check for assertions (will be extended, to e.g.,
address/undefined behavior sanitizer analysis)
- RetxCheck: retransmission check as used in the CI
- LastLineContains to check the last line containes a specific
string
- ContainsString to check a specific string appears in logs
- EndsWithBye: check that towards the end (but not necessarily the
last line), there is a Bye. string (a workaround since we don't
always terminate with 0)
- There is a plan to use return codes after stopping in docker, but most
executables do not end with 0 most of the time, such that we needed to
abandon this plan.
- Additional fixes (see commit messages) and delete unused code
GPU Channel Simulation: nr_dlsim/nr_ulsim Integration + CI Test
This MR is a follow-up to !3886 (merged) and extends CUDA-based channel
simulation support to nr_dlsim and nr_ulsim, together with the
corresponding CI tests.
The MR incorporates:
- Cherry-picked commits e0c204f7 and 5f344876 from !3588
- Integration of new CI physim tests using GPU-accelerated channel
simulation in nr_dlsim and nr_ulsim tests, see commit c727b3bb
New CI job: RAN-Channel-Simulation
https://jenkins-oai.eurecom.fr/job/RAN-Channel-Simulation/
[CI] Pull Ubuntu base image from internal registry
This MR updates the Docker build process for OAI images to use a private
registry for the Ubuntu base image ubuntu:noble. This will reduce the
possibility for CI disruption if no internet connection exists.
- Replace FROM ubuntu:noble with FROM <DEFAULT_REGISTRY>/ubuntu:noble in Ubuntu Dockerfiles
This change overcomes the pull rate limit issues from DockerHub in the
CI.
The <DEFAULT_REGISTRY> contains a multi-architecture image (arm64 and
amd64)
$ docker buildx imagetools inspect gracehopper3-oai.sboai.cs.eurecom.fr/ubuntu:noble | grep Platform
Platform: linux/amd64
Platform: linux/arm64
- The scripts run from the jenkins server everyday using a crontab.
- It pulls the AMD and ARM ubuntu:noble image on respective servers and
checks if the internal registry is in sync.
- If not, it pushes the ubuntu:noble image to the registry to remain in
sync with dockerhub.
It was initially foreseen to just check the program exit code, but most
executables don't return with 0 (so the CI would always fail). Add a
separate check that will check for the Bye., only going through the last
X lines instead of the entire file.
- we agreed to basically throw away the old log checker (as it is
unmaintainable)
- now: provide list of analysis to do
* all checkers in cls_log_analysis, can be unit tested (see also next
commit)
* looked up dynamically to simplify adding of new tests + config
* configured with
<analysis>
<services>DESCRIPTION [DESCRIPTION...]</services>
<service>DESCRIPTION</service>
...
</analysis>
where DESCRIPTION follows service=func[=options]:
- service is name of service to check
- func is function to call, which receives filename
- option are arbitary options to pass to func
<services> is whitespace delimited (so can take multiple service
analysis definitions)
<service> exists to allow service definitions with whitespace
changes that would not work in <services>
- I initially planned to check return code, but most softmodems actually
exit with non-zero return code, so this is still TODO
- Default analyzer (checking for assertions, ...) is always run if a
service is listed
- checks for file size
- add unit tests
- it is spaghetti code: difficult to understand which code paths are
taken
- the only effect seems to be in the UE log analysis function: it might
make the log fail. However, we test that iperf is successful
(Iperf2_Unidir()), and would detect before the log analysis if MBMS
traffic does not work
- the rest is only messages that likely nobody reads
The real reason for this change is to reduce the number of class members
in the ran.py class, to simplify the functioning of the eNB analysis
function (no self) to reduce dependencies on the ran.py class.
set -e makes the script exit on error. However, there is code to handle
a non-zero return code of the script, and -e prevents from writing the
footer.
add set -x to see all the commands that are being executed.
Use a raw string as suggested:
ci-scripts/cls_static_code_analysis.py:93: SyntaxWarning: invalid escape sequence '\:'
result = re.search('([a-zA-Z0-9\:\-\.\/])+\.git', self.ranRepository)
ci-scripts/cls_static_code_analysis.py:125: SyntaxWarning: invalid escape sequence '\.'
ret = re.search('cppcheck version="(?P<version>[0-9\.]+)"', str(line))
ci-scripts/cls_oai_html.py:221: SyntaxWarning: invalid escape sequence '\/'
cmd = "sed -i -e 's/__STATE_" + self.htmlTabNames[0] + "__/<span class=\"glyphicon glyphicon-remove\"><\/span>/' test_results.html"
ci-scripts/ran.py:428: SyntaxWarning: "\[" is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\\["? A raw string is also an option.
result = re.search('\[gNB [0-9]+\]\[RAPROC\] PUSCH with TC_RNTI 0x[0-9a-fA-F]+ received correctly, adding UE MAC Context RNTI 0x[0-9a-fA-F]+', str(line))
ci-scripts/ran.py:457: SyntaxWarning: "\[" is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\\["? A raw string is also an option.
result = re.search('\[PHY\]\s+problem receiving samples', str(line))
ci-scripts/ran.py:461: SyntaxWarning: "\[" is an invalid escape sequence. Such sequences will not work in the future. Did you mean "\\["? A raw string is also an option.
result = re.search('\[MAC\]\s+Removing UE', str(line))
I am not sure we ever need the millisecond output. The date is typically
also clear from the output (we could print the date at the beginning,
once, instead, but this is not done here. Finally, in Jenkins, we
typically get time stamps. Thus, it's not necessary to print this by
default.
On the other hand, since it might be useful when running locally, make
time stamps in logs configurable.
Since the grand-parent commit, if possible, the gNBs synchronize on a
slot-level boundary. Some configuration files/command line overrides
left the same SSB position for two DUs in RFsim-based handover
scenarios, which then makes that the UE cannot decode the SSB.
This commit changes the positions such that this is possible again. In
the F1 HO docker-compose
(ci-scripts/yaml_files/5g_f1_rfsimulator/docker-compose.yaml), we set it
to 2 to send only one SSB (not 2 with bitmap 3).
