Further features

This page gives an overview about two modules in WRaInfo, which are important for the management of raw data. At first we import wrainfo.

import wrainfo as wrf

[2]:

import datetime as dt

Error file list module

This module includes function to handle faulty data. Therefore we create an empty file list and in the next step we add faulty files to the file list.

[3]:

create_error_filelist(out_dir="/tests/data/error_flist/new_error_flist")

-- output to /tests/data/error_flist/new_error_flist

[3]:

True

Now we can add a faulty file to the created error file list. We have to define the exact time and in the directory in the configuration file. Updating the error file list, then you will be asked again if you really want to add the file to the error file list.

[4]:

path_to_config_file = "/tests/data/test_settings_wrainfo.json"

[5]:

wrf.error_flist.update_error_flist(start_time=dt.datetime(2022,2,16,13,40),
                                   end_time=dt.datetime(2022,2,16,13,50),
                                   path=path_to_config_file,
                                   pattern='_000.scnx.gz')

Would you like add this file to error flist?
/tests/data/raw/2022/02/16/2006_20220216_134500.h5
Confirm? [Y/N] Y
/tests/data/raw/2022/02/16/2006_20220216_134500.h5
successfully added to error file list.
Would you like save the changes to output directory?
Confirm? [Y/N] Y
output to
/tests/data/error_flist/new_error_flist

[5]:

True

And now we check, whether the filename is added to the error file list.

[6]:

wrf.reader.load_error_flist(path=path_to_config_file)

[6]:

['/tests/data/raw/2022/02/16/2006_20220216_134500.h5']

If we do not agree that the error file list should be changed the following happens. The file does not add to the error file list and we get the unchanged error file list as output.

[7]:

wrf.error_flist.update_error_flist(start_time=dt.datetime(2022,2,16,13,45),
                                   end_time=dt.datetime(2022,2,16,13,55),
                                   path=path_to_config_file,
                                   pattern='_000.scnx.gz')

Would you like add this file to error flist?
/misc/klima4/alicek/GitLab_repositories/wrainfo/tests/data/raw/2022/02/16/2006_20220216_134500.h5
Confirm? [Y/N] N
Would you like add this file to error flist?
/misc/klima4/alicek/GitLab_repositories/wrainfo/tests/data/raw/2022/02/16/2006_20220216_135000.h5
Confirm? [Y/N] N
Would you like save the changes to output directory?
Confirm? [Y/N] N

[7]:

['/tests/data/raw/2022/02/16/2006_20220216_134500.h5']

Library Reference

Seealso

Get more information about the error filelist module in the library reference section.

Compression module

This module of the package is useful to save the raw data of a day in a compressed file. The function loads all files of one day and add these to one tar.gz -file in the output directory. As option the original files can be deleted. With the function extract_files all compressed files will be extracted.

[8]:

path_to_config_file = "/tests/data/test_settings_wrainfo.json"

wrf.compression.compress_furuno_raw_data(start_time=dt.datetime(2022,2,16),
                                         end_time=dt.datetime(2022,2,17),
                                         path=path_to_config_file,
                                         delta=dt.timedelta(days=1),
                                         delete_files=False)

[8]:

True

Library Reference

Seealso

Get more information about the error compression module in the library reference section.