# fish **Repository Path**: martins3/fish ## Basic Information - **Project Name**: fish - **Description**: No description available - **Primary Language**: Unknown - **License**: Not specified - **Default Branch**: master - **Homepage**: None - **GVP Project**: No ## Statistics - **Stars**: 0 - **Forks**: 0 - **Created**: 2024-09-15 - **Last Updated**: 2024-09-15 ## Categories & Tags **Categories**: Uncategorized **Tags**: None ## README # 环境搭建 ## nixos 中搭建环境 nixos 中似乎遇到一点小问题 ```txt 🧀 pip3 install -r requirements.txt Collecting matplotlib==3.4.2 (from -r requirements.txt (line 1)) Downloading matplotlib-3.4.2.tar.gz (37.3 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 37.3/37.3 MB 1.1 MB/s eta 0:00:00 Installing build dependencies ... done Getting requirements to build wheel ... done Installing backend dependencies ... done Preparing metadata (pyproject.toml) ... error error: subprocess-exited-with-error × Preparing metadata (pyproject.toml) did not run successfully. │ exit code: 1 ╰─> [157 lines of output] :34: SetuptoolsDeprecationWarning: The test command is disabled and refere nces to it are deprecated. !! ************************************************************************** ****** Please remove any references to `setuptools.command.test` in all supported versions of the affected package. By 2024-Nov-15, you need to update your project and remove deprecated call s or your builds will no longer be supported. ************************************************************************** ****** !! /tmp/pip-build-env-q0rn_mlk/overlay/lib/python3.11/site-packages/setuptools/dist.p y:655: SetuptoolsDeprecationWarning: The namespace_packages parameter is deprecated. !! ************************************************************************** ****** Please replace its usage with implicit namespaces (PEP 420). See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword- namespace-packages for details. ************************************************************************** ****** !! ep.load()(self, ep.name, value) Edit setup.cfg to change the build options; suppress output with --quiet. BUILDING MATPLOTLIB matplotlib: yes [3.4.2] python: yes [3.11.9 (main, Apr 2 2024, 08:25:04) [GCC 13.2.0]] platform: yes [linux] tests: no [skipping due to configuration] macosx: no [Mac OS-X only] running dist_info creating /tmp/pip-modern-metadata-46d5i3et/matplotlib.egg-info writing /tmp/pip-modern-metadata-46d5i3et/matplotlib.egg-info/PKG-INFO writing dependency_links to /tmp/pip-modern-metadata-46d5i3et/matplotlib.egg-info/ dependency_links.txt writing namespace_packages to /tmp/pip-modern-metadata-46d5i3et/matplotlib.egg-inf o/namespace_packages.txt writing requirements to /tmp/pip-modern-metadata-46d5i3et/matplotlib.egg-info/requ ires.txt writing top-level names to /tmp/pip-modern-metadata-46d5i3et/matplotlib.egg-info/t op_level.txt writing manifest file '/tmp/pip-modern-metadata-46d5i3et/matplotlib.egg-info/SOURC ES.txt' /tmp/pip-build-env-q0rn_mlk/overlay/lib/python3.11/site-packages/setuptools/comman d/sdist.py:121: SetuptoolsDeprecationWarning: `build_py` command does not inherit from s etuptools' `build_py`. !! ************************************************************************** ****** Custom 'build_py' does not implement 'get_data_files_without_manifest'. Please extend command classes from setuptools instead of distutils. See https://peps.python.org/pep-0632/ for details. ************************************************************************** ****** !! self._add_data_files(self._safe_data_files(build_py)) Traceback (most recent call last): File "/tmp/pip-build-env-q0rn_mlk/normal/lib/python3.11/site-packages/numpy/_cor e/__init__.py", line 23, in from . import multiarray File "/tmp/pip-build-env-q0rn_mlk/normal/lib/python3.11/site-packages/numpy/_cor e/multiarray.py", line 10, in from . import overrides File "/tmp/pip-build-env-q0rn_mlk/normal/lib/python3.11/site-packages/numpy/_cor e/overrides.py", line 8, in from numpy._core._multiarray_umath import ( ImportError: libstdc++.so.6: cannot open shared object file: No such file or direc tory During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/tmp/pip-build-env-q0rn_mlk/normal/lib/python3.11/site-packages/numpy/__in it__.py", line 114, in from numpy.__config__ import show as show_config File "/tmp/pip-build-env-q0rn_mlk/normal/lib/python3.11/site-packages/numpy/__co nfig__.py", line 4, in from numpy._core._multiarray_umath import ( File "/tmp/pip-build-env-q0rn_mlk/normal/lib/python3.11/site-packages/numpy/_cor e/__init__.py", line 49, in raise ImportError(msg) ImportError: IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE! Importing the numpy C-extensions failed. This error can happen for many reasons, often due to issues with your setup or how NumPy was installed. We have compiled some common reasons and troubleshooting tips at: https://numpy.org/devdocs/user/troubleshooting-importerror.html Please note and check the following: * The Python version is: Python3.11 from "/nix/store/pfv4raslwhi3101k342752v65zx kwrxq-python3-3.11.9/bin/python3.11" * The NumPy version is: "2.1.1" and make sure that they are the versions you expect. Please carefully study the documentation linked above for further help. Original error was: libstdc++.so.6: cannot open shared object file: No such file o r directory The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/nix/store/v8m1r2jrfkmrzf48r4xzcszzf00f659b-python3-3.11.9-env/lib/python3 .11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in main() File "/nix/store/v8m1r2jrfkmrzf48r4xzcszzf00f659b-python3-3.11.9-env/lib/python3 .11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main json_out['return_val'] = hook(**hook_input['kwargs']) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/nix/store/v8m1r2jrfkmrzf48r4xzcszzf00f659b-python3-3.11.9-env/lib/python3 .11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 149, in prepare_metadata_for_build_wheel return hook(metadata_directory, config_settings) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/tmp/pip-build-env-q0rn_mlk/overlay/lib/python3.11/site-packages/setuptool s/build_meta.py", line 373, in prepare_metadata_for_build_wheel self.run_setup() File "/tmp/pip-build-env-q0rn_mlk/overlay/lib/python3.11/site-packages/setuptool s/build_meta.py", line 503, in run_setup super().run_setup(setup_script=setup_script) File "/tmp/pip-build-env-q0rn_mlk/overlay/lib/python3.11/site-packages/setuptool s/build_meta.py", line 318, in run_setup exec(code, locals()) File "", line 258, in File "/tmp/pip-build-env-q0rn_mlk/overlay/lib/python3.11/site-packages/setuptool s/__init__.py", line 117, in setup return distutils.core.setup(**attrs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/tmp/pip-build-env-q0rn_mlk/overlay/lib/python3.11/site-packages/setuptool s/_distutils/core.py", line 184, in setup return run_commands(dist) ^^^^^^^^^^^^^^^^^^ File "/tmp/pip-build-env-q0rn_mlk/overlay/lib/python3.11/site-packages/setuptool s/_distutils/core.py", line 200, in run_commands dist.run_commands() File "/tmp/pip-build-env-q0rn_mlk/overlay/lib/python3.11/site-packages/setuptool s/_distutils/dist.py", line 954, in run_commands self.run_command(cmd) File "/tmp/pip-build-env-q0rn_mlk/overlay/lib/python3.11/site-packages/setuptool s/dist.py", line 950, in run_command super().run_command(command) File "/tmp/pip-build-env-q0rn_mlk/overlay/lib/python3.11/site-packages/setuptool s/_distutils/dist.py", line 973, in run_command cmd_obj.run() File "/tmp/pip-build-env-q0rn_mlk/overlay/lib/python3.11/site-packages/setuptool s/command/dist_info.py", line 93, in run self.egg_info.run() File "/tmp/pip-build-env-q0rn_mlk/overlay/lib/python3.11/site-packages/setuptool s/command/egg_info.py", line 311, in run self.find_sources() File "/tmp/pip-build-env-q0rn_mlk/overlay/lib/python3.11/site-packages/setuptool s/command/egg_info.py", line 319, in find_sources mm.run() File "/tmp/pip-build-env-q0rn_mlk/overlay/lib/python3.11/site-packages/setuptool s/command/egg_info.py", line 540, in run self.add_defaults() File "/tmp/pip-build-env-q0rn_mlk/overlay/lib/python3.11/site-packages/setuptool s/command/egg_info.py", line 578, in add_defaults sdist.add_defaults(self) File "/tmp/pip-build-env-q0rn_mlk/overlay/lib/python3.11/site-packages/setuptool s/command/sdist.py", line 108, in add_defaults super().add_defaults() File "/tmp/pip-build-env-q0rn_mlk/overlay/lib/python3.11/site-packages/setuptool s/_distutils/command/sdist.py", line 250, in add_defaults self._add_defaults_ext() File "/tmp/pip-build-env-q0rn_mlk/overlay/lib/python3.11/site-packages/setuptool s/_distutils/command/sdist.py", line 334, in _add_defaults_ext build_ext = self.get_finalized_command('build_ext') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/tmp/pip-build-env-q0rn_mlk/overlay/lib/python3.11/site-packages/setuptool s/_distutils/cmd.py", line 303, in get_finalized_command cmd_obj.ensure_finalized() File "/tmp/pip-build-env-q0rn_mlk/overlay/lib/python3.11 s/_distutils/cmd.py", line 111, in ensure_finalized self.finalize_options() File "", line 90, in finalize_options File "", line 90, in File "/tmp/pip-install-k3cwfbar/matplotlib_5c9bcf1ced7a4 pext.py", line 383, in get_extensions add_numpy_flags(ext) File "/tmp/pip-install-k3cwfbar/matplotlib_5c9bcf1ced7a4 pext.py", line 498, in add_numpy_flags import numpy as np File "/tmp/pip-build-env-q0rn_mlk/normal/lib/python3.11/ it__.py", line 119, in raise ImportError(msg) from e ImportError: Error importing numpy: you should not try to its source directory; please exit the numpy source your python interpreter from there. [end of output] note: This error originates from a subprocess, and is likely n error: metadata-generation-failed × Encountered error while generating package metadata. ╰─> See above for output. note: This is an issue with the package mentioned above, not pip hint: See above for details. free_swim_illum_var on  main [!?] via 🐍 v3.11.9 via ❄️ impure (hello-env) 13900k took 45s 🤒 ``` ## conda 也遇到了问题 ```sh wget https://repo.anaconda.com/archive/Anaconda3-2024.06-1-Linux-x86_64.sh ./Anaconda3-2024.06-1-Linux-x86_64.sh ``` 但是按照 README 来执行，发现 conda 的 ```sh conda install --file requirements.txt ``` ### 先跳过版本尝试将所有依赖补齐 ## 无法执行 utils 模块简单的阅读一下 python 的语法: https://docs.python.org/3/tutorial/modules.html 将 plots.py 从 src 移动到 repo 的根目录下即可: ```txt 🧀 python3 plots.py ################ Creating a large dataset to gather multiple experiments ################ ///// WARNING : here we average all variables over 1 s \\\\ Traceback (most recent call last): File "/home/martins3/core/free_swim_illum_var/.venv/lib/python3.11/site-packages/xarray/core/concat.py", line 254, in concat first_obj, objs = utils.peek_at(objs) ^^^^^^^^^^^^^^^^^^^ File "/home/martins3/core/free_swim_illum_var/.venv/lib/python3.11/site-packages/xarray/core/utils.py", line 200, in peek_at peek = next(gen) ^^^^^^^^^ StopIteration During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/martins3/core/free_swim_illum_var/plots.py", line 173, in wrapper_plot(paths, path_timeserie) File "/home/martins3/core/free_swim_illum_var/plots.py", line 148, in wrapper_plot ds = dataloader_multiple(paths) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/martins3/core/free_swim_illum_var/utils/loader.py", line 59, in dataloader_multiple DS = xr.concat(loaded_ds, dim='experiment', combine_attrs='drop', compat='no_conflicts') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/martins3/core/free_swim_illum_var/.venv/lib/python3.11/site-packages/xarray/core/concat.py", line 256, in concat raise ValueError("must supply at least one object to concatenate") ValueError: must supply at least one object to concatenate free_swim_illum_var on  main [✘?] via 🐍 v3.11.9 (.venv) via ❄️ impure (hello-env) 13900k 🤒 ``` ## 发现没有数据，从百度网盘下载数据 ## 下载数据之后，执行，发现如下问题 ```txt ################ Creating a large dataset to gather multiple experiments ################ ///// WARNING : here we average all variables over 1 s \\\\ ['/home/martins3/core/fish/mi/3_VarLight/2022-01-04/1/', '/home/martins3/core/fish/mi/3_VarLight/2022-01-04/2/', '/home/martins3/cor e/fish/mi/3_VarLight/2022-01-06/1/', '/home/martins3/core/fish/mi/3_VarLight/2022-01-06/2/', '/home/martins3/core/fish/mi/3_VarLight /2021-12-21/1/', '/home/martins3/core/fish/mi/3_VarLight/2021-12-21/2/'] Merging file : cleaned/3_VarLight//home/martins3/core/fish/mi/3_VarLight/2022-01-04/1//trajectory.nc - 1/6 Traceback (most recent call last): File "/home/martins3/core/free_swim_illum_var/plots.py", line 173, in wrapper_plot(paths, path_timeserie) File "/home/martins3/core/free_swim_illum_var/plots.py", line 148, in wrapper_plot ds = dataloader_multiple(paths) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/martins3/core/free_swim_illum_var/utils/loader.py", line 47, in dataloader_multiple ds = dataloader(f'{path}/trajectory.nc') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/martins3/core/free_swim_illum_var/utils/loader.py", line 25, in dataloader ds = xr.open_dataset(data_file_name) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/martins3/core/free_swim_illum_var/.venv/lib/python3.11/site-packages/xarray/backends/api.py", line 592, in open_datase t engine = plugins.guess_engine(filename_or_obj) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/martins3/core/free_swim_illum_var/.venv/lib/python3.11/site-packages/xarray/backends/plugins.py", line 187, in guess_e ngine raise ValueError(error_msg) ValueError: found the following matches with the input file in xarray's IO backends: ['netcdf4', 'h5netcdf']. But their dependencies may not be installed, see: https://docs.xarray.dev/en/stable/user-guide/io.html https://docs.xarray.dev/en/stable/getting-started-guide/installing.html free_swim_illum_var on  main [»!?] via 🐍 v3.11.9 (.venv) via ❄️ impure (hello-env) 13900k 🤒 ``` 根据报错的提示，参考 https://docs.xarray.dev/en/stable/getting-started-guide/installing.html 中，按照如下内容，之后问题解决: ```sh python3 -m pip install "xarray[io]" # Install optional dependencies for handling I/O python3 -m pip install "xarray[accel]" # Install optional dependencies for accelerating xarray python3 -m pip install "xarray[parallel]" # Install optional dependencies for dask arrays python3 -m pip install "xarray[viz]" # Install optional dependencies for visualization python3 -m pip install "xarray[complete]" # Install all the above ``` ## 反思遇到的问题： 1. nixos 和普通系统不同，遇到了很多的动态库报错，不过关系不大，也没有参考意义，所以这里就不多说了。 - 如果实在是有兴趣，可以到 https://github.com/Martins3/My-Linux-Config/blob/master/docs/nix.md 中检查 2. 其实主要是 xarray 的加载问题，其文档就告诉，然后就可以解决 ## 需要补习的内容 1. python 的环境管理，建议使用 venv 来隔离各种发环境: ```txt python -m venv .venv # 构建虚拟环境，之后 pip3 install 的包都在这个目录 source .venv/bin/activate # 执行命令的时候会加载这个环境 ``` 2. shell 的基本使用，过多的依赖 pycharm 会搞不清楚整个项目是如何搭建的，可以更多的在 shell 中操作 - 例如上面的，source 命令到底是什么意思，具体参考 https://wangdoc.com/bash/ ，不容易，但是掌握了受益终生。以上都是指导方向，而不是具体的步骤。