B lc^@sUdZddlmZddlZddlZddlZddlmZddlm Z ddl Z ddl Z dZ de d<d e d <d d d dZedd ddZdd ddZdd ddZd ddddddZd!d d ddZdS)"z9Module providing access to all known dataset definitions.) annotationsN)Path)Anyzdict[str, Any] definitionzset[str]fileinfo_dirtyNone)returncCsiatatdd}tdd}x|dD]}|}|j}t |t|<t }| || t|d<||j}|st|q2t |}|dt|dkr|dtkr|t|d<q2t|q2WdS) z Read dataset .yml files from definitions/ and hashinfo/ directories. This is done once during the module import stage. dials_dataZ definitionshashinfoz*.ymlhashr formatversionN)rsetrimportlib_resourcesfilesZglobZ read_bytesZstemyamlZ safe_loadhashlibZsha256updateZ hexdigestnameexistsadd_hashinfo_formatversion)Zbase_directoryZhash_directoryZdefinition_fileZdataset_definition dataset_nameZdhashZh_filer r2lib/python3.7/site-packages/dials_data/datasets.py_load_yml_definitionss(    rcCst|dtdS)zT Generate a dictionary for the integrity information of a specific dataset. r )rr )rr)rrrrcreate_integrity_record7s rrc CstdrFy ttjd}|jddd|Stttfk rDYnXytd}|r\|SWntk rrYnXtdry$ttjdd}|jddd|Stttfk rYnXy"t dd}|jddd|Sttfk rt dYnXd S) aC Returns an appropriate location where the downloaded regression data should be stored. In order of evaluation: * If the environment variable DIALS_DATA is set and exists or can be created then use that location * If a Diamond Light Source specific path exists then use that location * If the environment variable LIBTBX_BUILD is set and the directory 'dials_data' exists or can be created underneath that location then use that. * Use ~/.cache/dials_data if it exists or can be created * Otherwise fail with a RuntimeError Z DIALS_DATAT)parentsexist_okz,/dls/science/groups/scisoft/DIALS/dials_dataZ LIBTBX_BUILDr z.cachezQCould not determine regression data location. Use environment variable DIALS_DATAN) osgetenvrenvironmkdirKeyError TypeErrorOSErroris_dirhome RuntimeError)Z repositoryrrrrepository_locationAs6  r)intcCs*|tkr dStddt|ddDS)Nrcss|]}|dVqdS)sizeNr).0itemrrr qsz$get_resident_size..r Zverify)rsumr)Zdsrrrget_resident_sizensr0Bfloatstr)numsuffixr cCsZxHdD]@}|dkr$|d||S|dkr>|d||S|d}qW|dd|S)N)kMG z.1fiz.0fg@Tr)r4r5Zunitrrr_human_readablets  r<Fc Csd}xt|D]}|r t|qt|}|tkr6d}n tt|}tdj|||dtdj|dd||dd d |d rtd j||d tdtj |d||dqWdS)Nz zunverified datasetz1{shortname}: {dataset[name]} ({size_information})) shortnamedatasetsize_informationz{indent}{author} ({license})authorzunknown authorlicensezunknown license)r@indentrAZurlz{indent}{dataset[url]})rBr>z {} description)initial_indentsubsequent_indent) sortedprintrrr<r0formatgettextwrapfill)Zds_listquietrBr=r>r?rrrlist_known_definitions~s2   rM)r1)F)__doc__Z __future__rrrrJZpathlibrtypingrrrr__annotations__rrr)r0r<rMrrrrs$     -