ASCAT PPF
☛ Position: Senior Software Developer
☛ Schedule: Apr. 2015 - Dec. 2015
☛ Client: EUMETSAT
☛ Team size: 1 people
Project description
- Port the software code base from AIX platform to Linux,
- Improve on the data processing pipeline to generate required auxiliary data on the fly instead of relying on undesired precomputation stages.
- Optimize the multithreading structure of the software to achieve better performances.
Check the Official EUMETSAT ASCAT page for more details on the satellite payload.
Responsibilities & Achievements
- I replaced AIX/xlC specific library (IBM ESSL library) with an open source multi-platform alternative (FFTW).
- I took platform endianess into account within file I/O operations.
- I updated the code base to support both gcc and xlC compilers.
- I conducted continuous regression tests against the reference implementation to validate changes in the algorithms.
- I built a tool (in LuaJIT) from scratches to read and compare the content of the ASCAT PPF outputs that was about 100 times faster than the default tool provided by EUMETSAT.
- I streamlined data processing pipeline, generating required preprocessing NTB data on-the-fly instead of using a dedicated stage.
- I unified the multithread handling systems in the software, replacing usage of OpenMP with a custom thread pool mechanism.
⇒ The tasks performed on the ASCAT PPF were all completed successfully and exceeded expectations by providing a significant improvement of the multithreaded performances: by reducing drastically the number of synchronization artefacts in the code, the software speed could now be scaled up further (up to a factor of 6 so far) by using all the cores available on the target stations (40 cores). This speed increase could not be achieved on the initial version as the speed gain degraded significantly when using more than 4 threads.
Skills & Techs
- C++
- AIX (eg. xlC/xlf compilers)
- Boost Threads
- Profiling tools
- Lua
- Latex
- Shell scripting