Events

Machine learning assumes working with large datasets. It is a common situation where the datasets do not fit into memory. To iterate and process datasets of arbitrary size, one needs a specific framework. In the seminar we discuss the key concepts of such frameworks that include data indexing, batch processing and pipelines construction. As a particular implementation we consider the framework https://github.com/observethesun/helio.git and show how it helps to implement unified pipelines for synoptic maps construction and coronal holes segmentation using SDO and SOHO data. We also discuss the questions related to Python code optimization and acceleration.