Zenn
Closed43

Open3Dのビルド

TensorFlow
CUDA
Open3D
PINTOPINTO
  • Ubuntu18.04
  • CUDA11.0
  • cuDNN 8.0
  • TensorFlow v2.4.1
  • Open3D v0.12.0
$ git clone -b v2.4.1 https://github.com/tensorflow/tensorflow.git
$ cd tensorflow
$ ./configure
$ sudo bazel build \
--config=cuda \
--config=noaws \
--config=nohdfs \
--config=nonccl \
--config=v2 \
--local_ram_resources=16384 \
--local_cpu_resources=6 \
//tensorflow/tools/pip_package:build_pip_package
PINTOPINTO

#error The version of CUB in your include path is not compatible with this release of Thrust. CUB is now included in the CUDA Toolkit, so you no longer need to use your own checkout of CUB. Define THRUST_IGNORE_CUB_VERSION_CHECK to ignore this.

PINTOPINTO
FROM nvidia/cuda:11.0.3-cudnn8-devel-ubuntu18.04

ENV OPEN3DVER=v0.12.0

# Install dependencies (1)
RUN apt-get update && apt-get install -y \
        automake autoconf libpng-dev nano wget npm \
        curl zip unzip libtool swig zlib1g-dev pkg-config \
        git wget xz-utils python3-mock libpython3-dev \
        libpython3-all-dev python3-pip g++ gcc make \
        pciutils cpio gosu git liblapack-dev liblapacke-dev

# Install dependencies (2)
#   Install TensorFlow v2.4.1 for CUDA11.0+cuDNN8.0, built from source code.
#   Because GLIBCXX_USE_CXX11_ABI needs to be turned on.
RUN pip3 install --upgrade pip \
    && pip3 install --upgrade onnx \
    && pip3 install --upgrade onnxruntime \
    && pip3 install --upgrade gdown \
    && pip3 install cmake==3.18.4 \
    && gdown --id 17GulM-XoRPJDEoU0CiM6zFcIMSPcXsOU \
    && pip3 install --force-reinstall tensorflow-2.4.1-cp36-cp36m-linux_x86_64.whl \
    && rm tensorflow-2.4.1-cp36-cp36m-linux_x86_64.whl \
    && find / -name "libtensorflow_framework*" \
    && pip3 install --upgrade pyyaml \
    && pip3 install --upgrade ninja \
    && pip3 install --upgrade yapf \
    && pip3 install --upgrade six \
    && pip3 install --upgrade wheel \
    && pip3 install --upgrade moc \
    # && pip3 install torch==1.7.1+cu110 \
    #                 torchvision==0.8.2+cu110 \
    #                 torchaudio===0.7.2 \
    #                 -f https://download.pytorch.org/whl/torch_stable.html \
    && ldconfig

# Build
#   Bypass CUDA and CUB version checks.
RUN git clone -b ${OPEN3DVER} --recursive https://github.com/intel-isl/Open3D \
    && cd Open3D \
    && git submodule update --init --recursive \
    && git clone -b master https://github.com/intel-isl/Open3D-ML \
    && chmod +x util/install_deps_ubuntu.sh \
    && sed -i 's/SUDO=${SUDO:=sudo}/SUDO=${SUDO:=}/g' \
              util/install_deps_ubuntu.sh \
    && util/install_deps_ubuntu.sh assume-yes \
    && sed -i -e "/^#ifndef THRUST_IGNORE_CUB_VERSION_CHECK$/i #define THRUST_IGNORE_CUB_VERSION_CHECK" \
                 /usr/local/cuda/targets/x86_64-linux/include/thrust/system/cuda/config.h \
    && cat /usr/local/cuda/targets/x86_64-linux/include/thrust/system/cuda/config.h \
    && mkdir build \
    && cd build \
    && cmake -DCMAKE_INSTALL_PREFIX=${HOME}/open3d \
             -DPYTHON_EXECUTABLE=$(which python3) \
             -DBUILD_PYTHON_MODULE=ON \
             -DBUILD_SHARED_LIBS=ON \
             -DBUILD_EXAMPLES=OFF \
             -DBUILD_UNIT_TESTS=OFF \
             -DBUILD_BENCHMARKS=ON \
             -DBUILD_CUDA_MODULE=ON \
             -DBUILD_CACHED_CUDA_MANAGER=ON \
             -DBUILD_GUI=ON \
             -DBUILD_JUPYTER_EXTENSION=ON \
             -DWITH_OPENMP=ON \
             -DWITH_IPPICV=ON \
             -DENABLE_HEADLESS_RENDERING=OFF \
             -DSTATIC_WINDOWS_RUNTIME=ON \
             -DGLIBCXX_USE_CXX11_ABI=ON \
             -DBUILD_RPC_INTERFACE=ON \
             -DUSE_BLAS=ON \
             -DBUILD_FILAMENT_FROM_SOURCE=OFF \
             -DWITH_FAISS=OFF \
             -DBUILD_LIBREALSENSE=ON \
             -DUSE_SYSTEM_LIBREALSENSE=OFF \
             -DBUILD_AZURE_KINECT=ON \
             -DBUILD_TENSORFLOW_OPS=ON \
             -DBUILD_PYTORCH_OPS=OFF \
             -DBUNDLE_OPEN3D_ML=ON \
             -DOPEN3D_ML_ROOT=../Open3D-ML \
            #  -DCMAKE_FIND_DEBUG_MODE=ON \
             ..

RUN cd /Open3D/build \
    && make -j$(nproc)

RUN cd /Open3D/build \
    # Only one of each of the following lines can be selected.
    #   (1) install -> Build and install the C++ shared library
    #   (2) install-pip-package -> Install pip-package
    #   (3) python-package -> Build the Python package
    #   (4) conda-package -> Building the conda package
    #   (5) pip-package -> Build the pip wheel file
    # && make install \
    # && make -j$(nproc) install-pip-package
    # && make -j$(nproc) python-package
    # && make -j$(nproc) conda-package
    && make -j$(nproc) pip-package

COPY entrypoint.sh /usr/local/bin/entrypoint.sh
RUN chmod +x /usr/local/bin/entrypoint.sh
ENTRYPOINT ["/usr/local/bin/entrypoint.sh"]
PINTOPINTO
-- THIRD_PARTY_DOWNLOAD_DIR is not specified, will download directly.
-- No CMAKE_BUILD_TYPE specified, default to RelWithDebInfo
-- The C compiler identification is GNU 7.5.0
-- The CXX compiler identification is GNU 7.5.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Open3D 0.12.0+313315d9
-- Using python from PYTHON_EXECUTABLE variable: /usr/bin/python3
-- (PYTHON_EXECUTABLE does NOT match python from PATH: /usr/bin/python)
-- Looking for a CUDA compiler
-- Looking for a CUDA compiler - /usr/local/cuda/bin/nvcc
-- CMAKE_CUDA_COMPILER: /usr/local/cuda/bin/nvcc
-- CMAKE_CUDA_COMPILER_VERSION: 
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE  
-- Found CUDA: /usr/local/cuda (found version "11.0") 
-- Building CUDA enabled
-- The CUDA compiler identification is NVIDIA 11.0.221
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Automatic GPU detection failed. Building for common architectures.
-- Autodetected CUDA architecture(s): 3.5;5.0;5.2;6.0;6.1;7.0;7.5;8.0;8.0+PTX
-- CUDA_GENCODES: -gencode arch=compute_35,code=sm_35 -gencode arch=compute_50,code=sm_50 -gencode arch=compute_52,code=sm_52 -gencode arch=compute_60,code=sm_60 -gencode arch=compute_61,code=sm_61 -gencode arch=compute_70,code=sm_70 -gencode arch=compute_75,code=sm_75 -gencode arch=compute_80,code=sm_80 -gencode arch=compute_80,code=compute_80
-- CMAKE_CUDA_TOOLKIT_INCLUDE_DIRECTORIES: /usr/local/cuda/targets/x86_64-linux/include
-- CMAKE_CUDA_IMPLICIT_INCLUDE_DIRECTORIES: /usr/include/c++/7;/usr/include/x86_64-linux-gnu/c++/7;/usr/include/c++/7/backward;/usr/lib/gcc/x86_64-linux-gnu/7/include;/usr/local/include;/usr/lib/gcc/x86_64-linux-gnu/7/include-fixed;/usr/include/x86_64-linux-gnu;/usr/include
-- Performing Test HAS_GLIBCXX_USE_CXX11_ABI
-- Performing Test HAS_GLIBCXX_USE_CXX11_ABI - Success
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Building library Assimp from source
-- Found OpenMP_C: -fopenmp (found version "4.5") 
-- Found OpenMP_CXX: -fopenmp (found version "4.5") 
-- Found OpenMP: TRUE (found version "4.5")  
-- Building with OpenMP
-- Building library 3rdparty_eigen3 from source
-- Building library 3rdparty_flann from source
-- Building library 3rdparty_nanoflann from source
-- Building library 3rdparty_glew from source
-- Building library 3rdparty_glfw3 from source
-- Could NOT find Vulkan (missing: VULKAN_LIBRARY VULKAN_INCLUDE_DIR) 
-- Using X11 for window creation
-- Found X11: /usr/include   
-- Building third-party library JPEG from source
-- Looking for a ASM_NASM compiler
-- Looking for a ASM_NASM compiler - NOTFOUND
-- NASM assembler not found - libjpeg-turbo performance may suffer
-- libturbojpeg: WITH_CRT_DLL=OFF
-- Building library 3rdparty_jsoncpp from source
-- Building library 3rdparty_lzf from source
-- Building library 3rdparty_tritriintersect from source
-- LIBUSB_LIB: /usr/lib/x86_64-linux-gnu/libusb-1.0.so
-- LIBUSB_INC: /usr/include/libusb-1.0
-- Building third-party library zlib from source
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of off64_t
-- Check size of off64_t - failed
-- Building third-party library libpng from source
-- Building library 3rdparty_rply from source
-- Building library 3rdparty_tinyfiledialogs from source
-- Building library 3rdparty_tinygltf from source
-- Building library 3rdparty_tinyobjloader from source
-- Building library 3rdparty_qhull_r from source
-- Building library 3rdparty_qhullcpp from source
-- Building library 3rdparty_fmt from source
-- Found PythonInterp: /usr/bin/python3 (found version "3.6.9") 
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython3.6m.so
-- Performing Test HAS_CPP14_FLAG
-- Performing Test HAS_CPP14_FLAG - Success
-- pybind11 v2.5.0
-- Building library 3rdparty_poisson from source
-- Found OpenGL: /usr/lib/x86_64-linux-gnu/libOpenGL.so   
-- Building library 3rdparty_imgui from source
-- Using prebuilt third-party library Filament
-- GLIBC version 2.27-3 found: Downloading Google Filament binary.
-- Filament is located at /Open3D/build/filament/src/ext_filament
-- CLANG_LIBDIR: /usr/lib/x86_64-linux-gnu
-- CPP_LIBRARY: /usr/lib/x86_64-linux-gnu/libc++.so
-- CPPABI_LIBRARY: /usr/lib/x86_64-linux-gnu/libc++abi.so
-- Boost source dir: /Open3D/build/boost/src/ext_boost
-- Looking for sgemm_
-- Looking for sgemm_ - not found
-- Looking for sgemm_
-- Looking for sgemm_ - found
-- Found BLAS: /usr/lib/x86_64-linux-gnu/libblas.so  
-- Looking for cheev_
-- Looking for cheev_ - not found
-- Looking for cheev_
-- Looking for cheev_ - found
-- A library with LAPACK API found.
-- A cache variable, namely LAPACKE_DIR, has been set to specify the install directory of LAPACKE
-- Looking for LAPACKE_dgeqrf
-- Looking for LAPACKE_dgeqrf - not found
-- Looking for lapacke : test with lapack fails
-- Looking for LAPACKE_dgeqrf
-- Looking for LAPACKE_dgeqrf - found
-- Found LAPACKE: /usr/lib/x86_64-linux-gnu/liblapacke.so  
-- Using system BLAS/LAPACK
-- Found CUB: /Open3D/3rdparty/cub  
-- Building TensorFlow ops
-- Building TensorFlow ops with CUDA
-- Using python from PYTHON_EXECUTABLE variable: /usr/bin/python3
-- (PYTHON_EXECUTABLE does NOT match python from PATH: /usr/bin/python)
-- Getting TensorFlow properties ...
2021-02-14 14:46:30.518633: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
-- TensorFlow       version: 2.4.1
--              include dir: /usr/local/lib/python3.6/dist-packages/tensorflow/include
--              library dir: /usr/local/lib/python3.6/dist-packages/tensorflow
--            framework lib: /usr/local/lib/python3.6/dist-packages/tensorflow/libtensorflow_framework.so
--              definitions: _GLIBCXX_USE_CXX11_ABI=1
--            use cxx11 abi: 1
-- TensorFlow matches Open3D ABI: 1 == ON
-- Found Tensorflow: /usr/local/lib/python3.6/dist-packages/tensorflow/include  
-- Found nanoflann: /Open3D/3rdparty/nanoflann/include  
-- Found parallelstl: /Open3D/3rdparty/parallelstl/include  
-- Found CUTLASS: /Open3D/3rdparty/cutlass  
-- Failed to find LLVM FileCheck
-- Found Git: /usr/bin/git (found version "2.17.1") 
-- git Version: v1.5.0-0ab2c290
-- Version: 1.5.0
-- Performing Test HAVE_CXX_FLAG_STD_CXX11
-- Performing Test HAVE_CXX_FLAG_STD_CXX11 - Success
-- Performing Test HAVE_CXX_FLAG_WALL
-- Performing Test HAVE_CXX_FLAG_WALL - Success
-- Performing Test HAVE_CXX_FLAG_WEXTRA
-- Performing Test HAVE_CXX_FLAG_WEXTRA - Success
-- Performing Test HAVE_CXX_FLAG_WSHADOW
-- Performing Test HAVE_CXX_FLAG_WSHADOW - Success
-- Performing Test HAVE_CXX_FLAG_WERROR
-- Performing Test HAVE_CXX_FLAG_WERROR - Success
-- Performing Test HAVE_CXX_FLAG_WSHORTEN_64_TO_32
-- Performing Test HAVE_CXX_FLAG_WSHORTEN_64_TO_32 - Failed
-- Performing Test HAVE_CXX_FLAG_FSTRICT_ALIASING
-- Performing Test HAVE_CXX_FLAG_FSTRICT_ALIASING - Success
-- Performing Test HAVE_CXX_FLAG_WNO_DEPRECATED_DECLARATIONS
-- Performing Test HAVE_CXX_FLAG_WNO_DEPRECATED_DECLARATIONS - Success
-- Performing Test HAVE_CXX_FLAG_WNO_DEPRECATED
-- Performing Test HAVE_CXX_FLAG_WNO_DEPRECATED - Success
-- Performing Test HAVE_CXX_FLAG_WSTRICT_ALIASING
-- Performing Test HAVE_CXX_FLAG_WSTRICT_ALIASING - Success
-- Performing Test HAVE_CXX_FLAG_WD654
-- Performing Test HAVE_CXX_FLAG_WD654 - Failed
-- Performing Test HAVE_CXX_FLAG_WTHREAD_SAFETY
-- Performing Test HAVE_CXX_FLAG_WTHREAD_SAFETY - Failed
-- Performing Test HAVE_CXX_FLAG_COVERAGE
-- Performing Test HAVE_CXX_FLAG_COVERAGE - Success
-- Performing Test HAVE_STD_REGEX
-- Performing Test HAVE_STD_REGEX
-- Performing Test HAVE_STD_REGEX -- success
-- Performing Test HAVE_GNU_POSIX_REGEX
-- Performing Test HAVE_GNU_POSIX_REGEX
-- Performing Test HAVE_GNU_POSIX_REGEX -- failed to compile
-- Performing Test HAVE_POSIX_REGEX
-- Performing Test HAVE_POSIX_REGEX
-- Performing Test HAVE_POSIX_REGEX -- success
-- Performing Test HAVE_STEADY_CLOCK
-- Performing Test HAVE_STEADY_CLOCK
-- Performing Test HAVE_STEADY_CLOCK -- success
-- npm found at: /usr/bin/npm. Jupyter support will be enabled.
-- BUILD_JUPYTER_EXTENSION is set to ON
-- Using Python version: 3.6
-- Using Python version major: 3
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
-- LTO enabled
-- PYPI_PACKAGE_NAME: open3d
-- Using python from PYTHON_EXECUTABLE variable: /usr/bin/python3
-- (PYTHON_EXECUTABLE does NOT match python from PATH: /usr/bin/python)
-- Getting TensorFlow properties ...
2021-02-14 14:46:46.821539: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
-- TensorFlow       version: 2.4.1
--              include dir: /usr/local/lib/python3.6/dist-packages/tensorflow/include
--              library dir: /usr/local/lib/python3.6/dist-packages/tensorflow
--            framework lib: /usr/local/lib/python3.6/dist-packages/tensorflow/libtensorflow_framework.so
--              definitions: _GLIBCXX_USE_CXX11_ABI=1
--            use cxx11 abi: 1
-- TensorFlow matches Open3D ABI: 1 == ON
-- ========================================================================
-- Open3D 0.12.0 Configuration Summary
-- ========================================================================
-- Enabled Features:
--   OpenMP .................................. ON
--   Headless Rendering ...................... OFF
--   Azure Kinect Support .................... ON
--   Intel RealSense Support ................. ON
--   CUDA Support ............................ ON
--   Build GUI ............................... ON
--   Build Shared Library .................... ON
--   Build Unit Tests ........................ OFF
--   Build Examples .......................... 
--   Build Python Module ..................... ON
--   - with Jupyter Notebook Support ......... ON
--   Build TensorFlow Ops .................... ON
--   Build PyTorch Ops ....................... OFF
--   Build Benchmarks ........................ ON
--   Bundle Open3D-ML ........................ ON
--   Build RPC interface ..................... ON
--   Force GLIBCXX_USE_CXX11_ABI= ............ 1
-- ========================================================================
-- Third-Party Dependencies:
--   EIGEN3 .................................. yes (build from source)
--   FAISS ................................... no
--   FILAMENT ................................ yes (build from source)
--   FLANN ................................... yes (build from source)
--   FMT ..................................... yes (build from source)
--   GLEW .................................... yes (build from source)
--   GLFW .................................... yes (build from source)
--   GOOGLETEST .............................. no
--   IMGUI ................................... yes (build from source)
--   JPEG .................................... yes (build from source)
--   JSONCPP ................................. yes (build from source)
--   LIBLZF .................................. yes (build from source)
--   OPENGL .................................. yes (build from source)
--   PNG ..................................... yes (build from source)
--   PYBIND11 ................................ yes (build from source)
--   QHULL ................................... yes (build from source)
--   LIBREALSENSE ............................ yes (build from source)
--   TINYFILEDIALOGS ......................... yes (build from source)
--   TINYGLTF ................................ yes (build from source)
--   TINYOBJLOADER ........................... yes (build from source)
-- ========================================================================
PINTOPINTO
[100%] Linking CXX shared module ../../lib/RelWithDebInfo/Python/cuda/pybind.cpython-36m-x86_64-linux-gnu.so
[100%] Built target pybind
Scanning dependencies of target python-package
[100%] Generating python ops.py
2021-02-16 14:46:27.648484: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2021-02-16 14:46:30.029417: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2021-02-16 14:46:30.439045: E tensorflow/core/lib/monitoring/collection_registry.cc:77] Cannot register 2 metrics with the same name: /tensorflow/cc/saved_model/load_attempt_count
2021-02-16 14:46:30.439075: E tensorflow/core/lib/monitoring/collection_registry.cc:77] Cannot register 2 metrics with the same name: /tensorflow/cc/saved_model/load_latency
2021-02-16 14:46:30.439098: E tensorflow/core/lib/monitoring/collection_registry.cc:77] Cannot register 2 metrics with the same name: /tensorflow/cc/saved_model/load_latency_by_stage
2021-02-16 14:46:30.444171: F tensorflow/core/framework/device_factory.cc:78] Duplicate registration of device factory for type GPU with the same priority 210
Aborted
make[3]: *** [lib/ml/tf/python/ops/ops.py] Error 134
cpp/pybind/CMakeFiles/python-package.dir/build.make:79: recipe for target 'lib/ml/tf/python/ops/ops.py' failed
CMakeFiles/Makefile2:2499: recipe for target 'cpp/pybind/CMakeFiles/python-package.dir/all' failed
make[2]: *** [cpp/pybind/CMakeFiles/python-package.dir/all] Error 2
make[1]: *** [cpp/pybind/CMakeFiles/pip-package.dir/rule] Error 2
CMakeFiles/Makefile2:2560: recipe for target 'cpp/pybind/CMakeFiles/pip-package.dir/rule' failed
make: *** [pip-package] Error 2
Makefile:898: recipe for target 'pip-package' failed
PINTOPINTO
cpp/pybind/CMakeFiles/python-package: lib/ml/tf/python/ops/ops.py
	cd /Open3D/build/cpp/pybind \
    && /usr/local/lib/python3.6/dist-packages/cmake/data/bin/cmake \
         -DPYTHON_PACKAGE_SRC_DIR=/Open3D/python \
         -DPYTHON_PACKAGE_DST_DIR=/Open3D/build/lib/python_package \
         -DPYTHON_VERSION=3.6 \
         "-DCOMPILED_MODULE_PATH_LIST=/Open3D/build/lib/RelWithDebInfo/Python/cuda/pybind.cpython-36m-x86_64-linux-gnu.so;/Open3D/build/lib/RelWithDebInfo/cuda/open3d_tf_ops.so" \
         "-DPYTHON_EXTRA_LIBRARIES=/usr/lib/x86_64-linux-gnu/libc++.so.1;/usr/lib/x86_64-linux-gnu/libc++abi.so" \
         -DBUILD_JUPYTER_EXTENSION=ON \
         -DBUILD_TENSORFLOW_OPS=ON \
         -DBUILD_PYTORCH_OPS=OFF \
         -DBUNDLE_OPEN3D_ML=ON \
         -DOPEN3D_ML_ROOT=/Open3D/Open3D-ML \
         -DBUILD_GUI=ON \
         -DGUI_RESOURCE_DIR=/Open3D/build/bin/resources \
         -DPROJECT_EMAIL=info@open3d.org \
         -DPROJECT_HOME=http://www.open3d.org \
         -DPROJECT_DOCS=http://www.open3d.org/docs \
         -DPROJECT_CODE=https://github.com/intel-isl/Open3D \
         -DPROJECT_ISSUES=https://github.com/intel-isl/Open3D/issues \
         -DPROJECT_VERSION=0.12.0+313315d9 \
         -DPROJECT_VERSION_THREE_NUMBER=0.12.0 \
         -DPYPI_PACKAGE_NAME=open3d \
         -P /Open3D/cpp/pybind/make_python_package.cmake
PINTOPINTO
$ docker build -t pinto0309/open3d-build:latest .
$ docker run --gpus all -it --rm \
    -e LOCAL_UID=$(id -u $USER) \
    -e LOCAL_GID=$(id -g $USER) \
    pinto0309/open3d-build:latest bash
PINTOPINTO

/Open3D/build/CMakeFiles/Makefile2

# Target rules for target cpp/pybind/CMakeFiles/pip-package.dir

# All Build rule for target.
cpp/pybind/CMakeFiles/pip-package.dir/all: cpp/pybind/CMakeFiles/python-package.dir/all
	$(MAKE) $(MAKESILENT) -f cpp/pybind/CMakeFiles/pip-package.dir/build.make cpp/pybind/CMakeFiles/pip-package.dir/depend
	$(MAKE) $(MAKESILENT) -f cpp/pybind/CMakeFiles/pip-package.dir/build.make cpp/pybind/CMakeFiles/pip-package.dir/build
	@$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --progress-dir=/Open3D/build/CMakeFiles --progress-num= "Built target pip-package"
.PHONY : cpp/pybind/CMakeFiles/pip-package.dir/all

# Build rule for subdir invocation for target.
cpp/pybind/CMakeFiles/pip-package.dir/rule: cmake_check_build_system
	$(CMAKE_COMMAND) -E cmake_progress_start /Open3D/build/CMakeFiles 85
	$(MAKE) $(MAKESILENT) -f CMakeFiles/Makefile2 cpp/pybind/CMakeFiles/pip-package.dir/all
	$(CMAKE_COMMAND) -E cmake_progress_start /Open3D/build/CMakeFiles 0
.PHONY : cpp/pybind/CMakeFiles/pip-package.dir/rule

# Convenience name for target.
pip-package: cpp/pybind/CMakeFiles/pip-package.dir/rule

.PHONY : pip-package

# clean rule for target.
cpp/pybind/CMakeFiles/pip-package.dir/clean:
	$(MAKE) $(MAKESILENT) -f cpp/pybind/CMakeFiles/pip-package.dir/build.make cpp/pybind/CMakeFiles/pip-package.dir/clean
.PHONY : cpp/pybind/CMakeFiles/pip-package.dir/clean
PINTOPINTO
./build/cpp/pybind/CMakeFiles/pip-package.dir/build.make
./build/cpp/pybind/CMakeFiles/python-package.dir/build.make
PINTOPINTO

./build/cpp/pybind/CMakeFiles/pip-package.dir/build.make

# CMAKE generated file: DO NOT EDIT!
# Generated by "Unix Makefiles" Generator, CMake Version 3.18

# Delete rule output on recipe failure.
.DELETE_ON_ERROR:


#=============================================================================
# Special targets provided by cmake.

# Disable implicit rules so canonical targets will work.
.SUFFIXES:


# Disable VCS-based implicit rules.
% : %,v


# Disable VCS-based implicit rules.
% : RCS/%


# Disable VCS-based implicit rules.
% : RCS/%,v


# Disable VCS-based implicit rules.
% : SCCS/s.%


# Disable VCS-based implicit rules.
% : s.%


.SUFFIXES: .hpux_make_needs_suffix_list


# Command-line flag to silence nested $(MAKE).
$(VERBOSE)MAKESILENT = -s

#Suppress display of executed commands.
$(VERBOSE).SILENT:

# A target that is always out of date.
cmake_force:

.PHONY : cmake_force

#=============================================================================
# Set environment variables for the build.

# The shell in which to execute make rules.
SHELL = /bin/sh

# The CMake executable.
CMAKE_COMMAND = /usr/local/lib/python3.6/dist-packages/cmake/data/bin/cmake

# The command to remove a file.
RM = /usr/local/lib/python3.6/dist-packages/cmake/data/bin/cmake -E rm -f

# Escaping for special characters.
EQUALS = =

# The top-level source directory on which CMake was run.
CMAKE_SOURCE_DIR = /Open3D

# The top-level build directory on which CMake was run.
CMAKE_BINARY_DIR = /Open3D/build

# Utility rule file for pip-package.

# Include the progress variables for this target.
include cpp/pybind/CMakeFiles/pip-package.dir/progress.make

cpp/pybind/CMakeFiles/pip-package:
	cd /Open3D/build/lib/python_package && /usr/bin/python3 setup.py bdist_wheel --dist-dir pip_package
	cd /Open3D/build/lib/python_package && echo pip\ wheel\ created\ at\ /Open3D/build/lib/python_package/pip_package

pip-package: cpp/pybind/CMakeFiles/pip-package
pip-package: cpp/pybind/CMakeFiles/pip-package.dir/build.make

.PHONY : pip-package

# Rule to build all files generated by this target.
cpp/pybind/CMakeFiles/pip-package.dir/build: pip-package

.PHONY : cpp/pybind/CMakeFiles/pip-package.dir/build

cpp/pybind/CMakeFiles/pip-package.dir/clean:
	cd /Open3D/build/cpp/pybind && $(CMAKE_COMMAND) -P CMakeFiles/pip-package.dir/cmake_clean.cmake
.PHONY : cpp/pybind/CMakeFiles/pip-package.dir/clean

cpp/pybind/CMakeFiles/pip-package.dir/depend:
	cd /Open3D/build && $(CMAKE_COMMAND) -E cmake_depends "Unix Makefiles" /Open3D /Open3D/cpp/pybind /Open3D/build /Open3D/build/cpp/pybind /Open3D/build/cpp/pybind/CMakeFiles/pip-package.dir/DependInfo.cmake --color=$(COLOR)
.PHONY : cpp/pybind/CMakeFiles/pip-package.dir/depend
PINTOPINTO

./build/cpp/pybind/CMakeFiles/python-package.dir/build.make

# CMAKE generated file: DO NOT EDIT!
# Generated by "Unix Makefiles" Generator, CMake Version 3.18

# Delete rule output on recipe failure.
.DELETE_ON_ERROR:


#=============================================================================
# Special targets provided by cmake.

# Disable implicit rules so canonical targets will work.
.SUFFIXES:


# Disable VCS-based implicit rules.
% : %,v


# Disable VCS-based implicit rules.
% : RCS/%


# Disable VCS-based implicit rules.
% : RCS/%,v


# Disable VCS-based implicit rules.
% : SCCS/s.%


# Disable VCS-based implicit rules.
% : s.%


.SUFFIXES: .hpux_make_needs_suffix_list


# Command-line flag to silence nested $(MAKE).
$(VERBOSE)MAKESILENT = -s

#Suppress display of executed commands.
$(VERBOSE).SILENT:

# A target that is always out of date.
cmake_force:

.PHONY : cmake_force

#=============================================================================
# Set environment variables for the build.

# The shell in which to execute make rules.
SHELL = /bin/sh

# The CMake executable.
CMAKE_COMMAND = /usr/local/lib/python3.6/dist-packages/cmake/data/bin/cmake

# The command to remove a file.
RM = /usr/local/lib/python3.6/dist-packages/cmake/data/bin/cmake -E rm -f

# Escaping for special characters.
EQUALS = =

# The top-level source directory on which CMake was run.
CMAKE_SOURCE_DIR = /Open3D

# The top-level build directory on which CMake was run.
CMAKE_BINARY_DIR = /Open3D/build

# Utility rule file for python-package.

# Include the progress variables for this target.
include cpp/pybind/CMakeFiles/python-package.dir/progress.make

cpp/pybind/CMakeFiles/python-package: lib/ml/tf/python/ops/ops.py
	cd /Open3D/build/cpp/pybind && /usr/local/lib/python3.6/dist-packages/cmake/data/bin/cmake -DPYTHON_PACKAGE_SRC_DIR=/Open3D/python -DPYTHON_PACKAGE_DST_DIR=/Open3D/build/lib/python_package -DPYTHON_VERSION=3.6 "-DCOMPILED_MODULE_PATH_LIST=/Open3D/build/lib/RelWithDebInfo/Python/cuda/pybind.cpython-36m-x86_64-linux-gnu.so;/Open3D/build/lib/RelWithDebInfo/cuda/open3d_tf_ops.so" "-DPYTHON_EXTRA_LIBRARIES=/usr/lib/x86_64-linux-gnu/libc++.so.1;/usr/lib/x86_64-linux-gnu/libc++abi.so" -DBUILD_JUPYTER_EXTENSION=ON -DBUILD_TENSORFLOW_OPS=ON -DBUILD_PYTORCH_OPS=OFF -DBUNDLE_OPEN3D_ML=ON -DOPEN3D_ML_ROOT=/Open3D/Open3D-ML -DBUILD_GUI=ON -DGUI_RESOURCE_DIR=/Open3D/build/bin/resources -DPROJECT_EMAIL=info@open3d.org -DPROJECT_HOME=http://www.open3d.org -DPROJECT_DOCS=http://www.open3d.org/docs -DPROJECT_CODE=https://github.com/intel-isl/Open3D -DPROJECT_ISSUES=https://github.com/intel-isl/Open3D/issues -DPROJECT_VERSION=0.12.0+313315d9 -DPROJECT_VERSION_THREE_NUMBER=0.12.0 -DPYPI_PACKAGE_NAME=open3d -P /Open3D/cpp/pybind/make_python_package.cmake

lib/ml/tf/python/ops/ops.py: lib/RelWithDebInfo/cuda/open3d_tf_ops.so
	@$(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --blue --bold --progress-dir=/Open3D/build/CMakeFiles --progress-num=$(CMAKE_PROGRESS_1) "Generating python ops.py"
	cd /Open3D/cpp/pybind && /usr/bin/python3 generate_tf_ops_wrapper.py --input /Open3D/python/open3d/ml/tf/python/ops/ops.py.in --output /Open3D/build/lib/ml/tf/python/ops/ops.py --lib /Open3D/build/lib/RelWithDebInfo/cuda/open3d_tf_ops.so

python-package: cpp/pybind/CMakeFiles/python-package
python-package: lib/ml/tf/python/ops/ops.py
python-package: cpp/pybind/CMakeFiles/python-package.dir/build.make

.PHONY : python-package

# Rule to build all files generated by this target.
cpp/pybind/CMakeFiles/python-package.dir/build: python-package

.PHONY : cpp/pybind/CMakeFiles/python-package.dir/build

cpp/pybind/CMakeFiles/python-package.dir/clean:
	cd /Open3D/build/cpp/pybind && $(CMAKE_COMMAND) -P CMakeFiles/python-package.dir/cmake_clean.cmake
.PHONY : cpp/pybind/CMakeFiles/python-package.dir/clean

cpp/pybind/CMakeFiles/python-package.dir/depend:
	cd /Open3D/build && $(CMAKE_COMMAND) -E cmake_depends "Unix Makefiles" /Open3D /Open3D/cpp/pybind /Open3D/build /Open3D/build/cpp/pybind /Open3D/build/cpp/pybind/CMakeFiles/python-package.dir/DependInfo.cmake --color=$(COLOR)
.PHONY : cpp/pybind/CMakeFiles/python-package.dir/depend
PINTOPINTO

https://github.com/intel-isl/Open3D/blob/master/cpp/pybind/CMakeLists.txt

# add additional optional compiled modules
if ( BUILD_TENSORFLOW_OPS )
    list( APPEND COMPILED_MODULE_PATH_LIST $<TARGET_FILE:open3d_tf_ops> )
    add_custom_command( OUTPUT "${CMAKE_BINARY_DIR}/lib/ml/tf/python/ops/ops.py"
            COMMAND ${PYTHON_EXECUTABLE} generate_tf_ops_wrapper.py --input "${PYTHON_PACKAGE_SRC_DIR}/open3d/ml/tf/python/ops/ops.py.in" --output "${CMAKE_BINARY_DIR}/lib/ml/tf/python/ops/ops.py" --lib $<TARGET_FILE:open3d_tf_ops>
PINTOPINTO

generate_tf_ops_wrapper.py

"""This script inspects the open3d_tf_ops library and generates function wrappers"""
import os
import sys
import inspect
import argparse
import textwrap
import tensorflow as tf
from yapf.yapflib.yapf_api import FormatFile

INDENT_SPACES = '    '

FN_TEMPLATE_STR = '''
def {fn_name_short}({fn_args}):
{docstring}
    return {fn_name}({args_fwd})

'''


def main():
    parser = argparse.ArgumentParser(description="Creates the ops.py file")
    parser.add_argument("--input",
                        type=str,
                        required=True,
                        help="input file with header")
    parser.add_argument("--output", type=str, required=True, help="output file")
    parser.add_argument("--lib",
                        type=str,
                        required=True,
                        help="path to open3d_tf_ops.so")

    args = parser.parse_args()
    print(args)

    oplib = tf.load_op_library(args.lib)

    generated_function_strs = ''
    for fn_name, value in inspect.getmembers(oplib):
        if not inspect.isfunction(value) or not fn_name.startswith(
                'open3d_') or fn_name.endswith('_eager_fallback'):
            continue

        docstring = getattr(oplib, fn_name).__doc__
        docstring = '"""' + docstring + '\n"""'
        docstring = textwrap.indent(docstring, INDENT_SPACES)

        signature = inspect.signature(value)

        fn_args = []
        args_fwd = []
        for _, param in signature.parameters.items():
            tmp = param.name
            if param.default != inspect.Parameter.empty:
                if isinstance(param.default, str):
                    tmp += '="{}"'.format(str(param.default))
                else:
                    tmp += '={}'.format(str(param.default))

            fn_args.append(tmp)
            args_fwd.append('{arg}={arg}'.format(arg=param.name))
        fn_args = ', '.join(fn_args)
        args_fwd = ', '.join(args_fwd)
        generated_function_strs += FN_TEMPLATE_STR.format(
            fn_name_short=fn_name[7:],
            fn_name='_lib.' + fn_name,
            fn_args=fn_args,
            docstring=docstring,
            args_fwd=args_fwd)

    with open(args.input, 'r') as f:
        input_header = f.read()

    os.makedirs(os.path.dirname(args.output), exist_ok=True)
    with open(args.output, 'w') as f:
        f.write(input_header + generated_function_strs)
    FormatFile(args.output, in_place=True)

    return 0


if __name__ == '__main__':
    sys.exit(main())
PINTOPINTO

ops.py.in

# ----------------------------------------------------------------------------
# -                        Open3D: www.open3d.org                            -
# ----------------------------------------------------------------------------
# The MIT License (MIT)
#
# Copyright (c) 2020 www.open3d.org
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
# FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
# IN THE SOFTWARE.
# ----------------------------------------------------------------------------

# This file is machine generated. Do not modify.
from .lib import _lib
PINTOPINTO
2021-02-17 13:56:12.077521: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2021-02-17 13:56:14.332272: I tensorflow/stream_executor/platform/default/dso_loader.cc:49] Successfully opened dynamic library libcudart.so.11.0
2021-02-17 13:56:14.714222: E tensorflow/core/lib/monitoring/collection_registry.cc:77] Cannot register 2 metrics with the same name: /tensorflow/cc/saved_model/load_attempt_count
2021-02-17 13:56:14.714249: E tensorflow/core/lib/monitoring/collection_registry.cc:77] Cannot register 2 metrics with the same name: /tensorflow/cc/saved_model/load_latency
2021-02-17 13:56:14.714266: E tensorflow/core/lib/monitoring/collection_registry.cc:77] Cannot register 2 metrics with the same name: /tensorflow/cc/saved_model/load_latency_by_stage
2021-02-17 13:56:14.718861: F tensorflow/core/framework/device_factory.cc:78] Duplicate registration of device factory for type GPU with the same priority 210
Aborted
make[3]: *** [lib/ml/tf/python/ops/ops.py] Error 134
cpp/pybind/CMakeFiles/python-package.dir/build.make:79: recipe for target 'lib/ml/tf/python/ops/ops.py' failed
make[2]: *** [cpp/pybind/CMakeFiles/python-package.dir/all] Error 2
CMakeFiles/Makefile2:2499: recipe for target 'cpp/pybind/CMakeFiles/python-package.dir/all' failed
make[1]: *** [cpp/pybind/CMakeFiles/pip-package.dir/rule] Error 2
CMakeFiles/Makefile2:2560: recipe for target 'cpp/pybind/CMakeFiles/pip-package.dir/rule' failed
make: *** [pip-package] Error 2
Makefile:898: recipe for target 'pip-package' failed
PINTOPINTO
$ pip3 install ninja
$ git clone -b v1.7.1 --recursive https://github.com/pytorch/pytorch
$ cd pytorch
$ sudo python3 setup.py build
$ sudo python3 setup.py bdist_wheel

$ git clone -b v0.8.2 https://github.com/pytorch/vision.git
$ cd vision
$ sudo python3 setup.py build
$ sudo python3 setup.py bdist_wheel

$ git clone -b v0.7.2 https://github.com/pytorch/audio.git
$ cd audio
$ sudo python3 setup.py build
$ sudo python3 setup.py bdist_wheel
PINTOPINTO
In file included from /Open3D/cpp/open3d/core/linalg/MatmulCPU.cpp:27:0:
/Open3D/cpp/open3d/core/linalg/BlasWrapper.h:37:22: error: variable or field 'gemm_cpu' declared void
 inline void gemm_cpu(CBLAS_LAYOUT layout,
                      ^~~~~~~~~~~~
/Open3D/cpp/open3d/core/linalg/BlasWrapper.h:37:22: error: 'CBLAS_LAYOUT' was not declared in this scope
/Open3D/cpp/open3d/core/linalg/BlasWrapper.h:37:22: note: suggested alternative: 'CBLAS_UPLO'
 inline void gemm_cpu(CBLAS_LAYOUT layout,
                      ^~~~~~~~~~~~
                      CBLAS_UPLO
/Open3D/cpp/open3d/core/linalg/BlasWrapper.h:38:38: error: expected primary-expression before 'trans_A'
                      CBLAS_TRANSPOSE trans_A,
                                      ^~~~~~~
/Open3D/cpp/open3d/core/linalg/BlasWrapper.h:39:38: error: expected primary-expression before 'trans_B'
                      CBLAS_TRANSPOSE trans_B,
                                      ^~~~~~~
/Open3D/cpp/open3d/core/linalg/BlasWrapper.h:40:44: error: expected primary-expression before 'm'
                      OPEN3D_CPU_LINALG_INT m,
                                            ^
/Open3D/cpp/open3d/core/linalg/BlasWrapper.h:41:44: error: expected primary-expression before 'n'
                      OPEN3D_CPU_LINALG_INT n,
                                            ^
/Open3D/cpp/open3d/core/linalg/BlasWrapper.h:42:44: error: expected primary-expression before 'k'
                      OPEN3D_CPU_LINALG_INT k,
                                            ^
/Open3D/cpp/open3d/core/linalg/BlasWrapper.h:43:31: error: expected primary-expression before 'alpha'
                      scalar_t alpha,
                               ^~~~~
/Open3D/cpp/open3d/core/linalg/BlasWrapper.h:44:22: error: expected primary-expression before 'const'
                      const scalar_t *A_data,
                      ^~~~~
PINTOPINTO
$ sudo dpkg -i sdkmanager_1.4.0-7363_amd64.deb

https://developer.nvidia.com/embedded/downloads

$ cd ~
$ mkdir -p nvidia_sdk/JetPack_4.5_Linux_JETSON_NANO_DEVKIT && \
  cd nvidia_sdk/JetPack_4.5_Linux_JETSON_NANO_DEVKIT
$ wget https://developer.nvidia.com/embedded/L4T/r32_Release_v5.0/sources/T210/public_sources.tbz2
$ tar -xvjf public_sources.tbz2 && rm public_sources.tbz2
$ cd Linux_for_Tegra/source/public
$ tar -xvjf kernel_src.tbz2
PINTOPINTO

Jetson Nanoのクロスコンパイル環境を作る
Jetson Nano Developer Kit SD Card Image
https://developer.download.nvidia.com/embedded/L4T/r32_Release_v5.0/Jetson_Nano/jetson-nano-jp45-sd-card-image.zip

$ cd ~
$ mkdir -p jetson_nano && cd jetson_nano
# Download SD card image - 14.3GB
$ wget https://developer.nvidia.com/jetson-nano-sd-card-image \
  -O jetson-nano-jp45-sd-card-image.zip
$ unzip  jetson-nano-jp45-sd-card-image.zip && \
  rm jetson-nano-jp45-sd-card-image.zip
$ su --preserve-environment

# fdisk sd-blob-b01.img

fdisk (util-linux 2.31.1) へようこそ。
ここで設定した内容は、書き込みコマンドを実行するまでメモリのみに保持されます。
書き込みコマンドを使用する際は、注意して実行してください。

コマンド (m でヘルプ):p
ディスク sd-blob-b01.img: 13.3 GiB, 14270070784 バイト, 27871232 セクタ
単位: セクタ (1 * 512 = 512 バイト)
セクタサイズ (論理 / 物理): 512 バイト / 512 バイト
I/O サイズ (最小 / 推奨): 512 バイト / 512 バイト
ディスクラベルのタイプ: gpt
ディスク識別子: 991E26BF-720A-4C06-A4B8-B8720A44C769

デバイス          開始位置 最後から   セクタ サイズ タイプ
sd-blob-b01.img1     28672 27867135 27838464  13.3G Linux ファイルシステム
sd-blob-b01.img2      2048     2303      256   128K Linux ファイルシステム
sd-blob-b01.img3      4096     4991      896   448K Linux ファイルシステム
sd-blob-b01.img4      6144     7295     1152   576K Linux ファイルシステム
sd-blob-b01.img5      8192     8319      128    64K Linux ファイルシステム
sd-blob-b01.img6     10240    10623      384   192K Linux ファイルシステム
sd-blob-b01.img7     12288    13055      768   384K Linux ファイルシステム
sd-blob-b01.img8     14336    14463      128    64K Linux ファイルシステム
sd-blob-b01.img9     16384    17279      896   448K Linux ファイルシステム
sd-blob-b01.img10    18432    19327      896   448K Linux ファイルシステム
sd-blob-b01.img11    20480    22015     1536   768K Linux ファイルシステム
sd-blob-b01.img12    22528    22655      128    64K Linux ファイルシステム
sd-blob-b01.img13    24576    24959      384   192K Linux ファイルシステム
sd-blob-b01.img14    26624    26879      256   128K Linux ファイルシステム

パーティション情報の項目がディスクの順序と一致しません。

コマンド (m でヘルプ): q
PINTOPINTO

offset = 28672 * 512 = 14680064

# mount -o loop,offset=14680064 sd-blob-b01.img /mnt
# ls -l /mnt

-rw-rw-rw-   1 root root    62  1月 16 07:54 README.txt
drwxr-xr-x   2 root root  4096 12月  1 13:49 bin
drwxr-xr-x   4 root root  4096  1月 20 05:21 boot
drwxr-xr-x   2 root root  4096  1月 20 05:58 dev
drwxr-xr-x 145 root root 12288  1月 20 05:59 etc
drwxr-xr-x   2 root root  4096  4月 24  2018 home
drwxr-xr-x  21 root root  4096  1月 20 05:21 lib
drwx------   2 root root 16384  1月 20 05:59 lost+found
drwxr-xr-x   2 root root  4096  8月  6  2018 media
drwxr-xr-x   2 root root  4096  4月 27  2018 mnt
drwxr-xr-x   4 root root  4096  1月 20 05:21 opt
drwxr-xr-x   2 root root  4096  4月 24  2018 proc
drwx------   2 root root  4096  4月 27  2018 root
drwxr-xr-x  17 root root  4096 12月  1 13:52 run
drwxr-xr-x   2 root root  4096 12月  1 13:50 sbin
drwxr-xr-x   2 root root  4096  5月 11  2018 snap
drwxr-xr-x   2 root root  4096  4月 27  2018 srv
drwxr-xr-x   2 root root  4096  4月 24  2018 sys
drwxrwxrwt   2 root root  4096  1月 20 05:58 tmp
drwxr-xr-x  12 root root  4096  1月 20 05:54 usr
drwxr-xr-x  19 root root  4096  1月 20 05:56 var

# cp /mnt/* ${HOME}/jetson_nano -rapvf
# apt update && apt install qemu-user-static

# cp /usr/bin/qemu-arm-static usr/bin/
# cp /usr/bin/qemu-aarch64-static usr/bin/

# rm sd-blob-b01.img
PINTOPINTO

ドライブ簡易マウント用スクリプト ch-mount.sh の作成

#!/bin/bash
sudo cp /usr/bin/qemu-arm-static usr/bin/
sudo cp /usr/bin/qemu-aarch64-static usr/bin/

function mnt() {
    echo "MOUNTING"
    sudo mount -t proc /proc ${2}/proc
    sudo mount -t sysfs /sys ${2}/sys    
    sudo mount -o bind /dev ${2}/dev
    sudo mount -o bind /run ${2}/run 
    #sudo mount --bind / ${2}/host
    #sudo mount -vt tmpfs shm ${2}/dev/shm
    #sudo mount -t /dev/shm ${2}/dev/shm
    sudo chroot ${2}
}

function umnt() {
    echo "UNMOUNTING"
    sudo umount ${2}/proc
    sudo umount ${2}/sys
    #sudo umount ${2}/dev/shm
    sudo umount ${2}/dev
    sudo umount ${2}/run
    #sudo umount ${2}/host
}


if [ "$1" == "-m" ] && [ -n "$2" ] ;
then
    mnt $1 $2
elif [ "$1" == "-u" ] && [ -n "$2" ];
then
    umnt $1 $2
else
    echo ""
    echo "Either 1'st, 2'nd or both parameters were missing"
    echo ""
    echo "1'st parameter can be one of these: -m(mount) OR -u(umount)"
    echo "2'nd parameter is the full path of rootfs directory(with trailing '/')"
    echo ""
    echo "For example: ch-mount -m /media/sdcard"
    echo ""
    echo 1st parameter : ${1}
    echo 2nd parameter : ${2}
fi

# chmod +x ch-mount.sh
# exit
$
PINTOPINTO

chrootの開始

$ bash ch-mount.sh -m jetson_nano

chrootの終了

# exit

アンマウント

$ bash ch-mount.sh -u jetson_nano
PINTOPINTO

chroot後のアーキテクチャの確認

# uname -a
Linux ubuntu 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 aarch64 aarch64 aarch64 GNU/Linux
PINTOPINTO

nvccへのパス通し
http://www.neko.ne.jp/~freewing/raspberry_pi/nvidia_jetson_nano_nvcc_not_found/

# find / -name nvcc
/usr/local/cuda-10.2/bin/nvcc

# ls -l /usr/local/
drwxr-xr-x  2 root root 4096 Apr 26  2018 bin
lrwxrwxrwx  1 root root    9 Jan 19 20:37 cuda -> cuda-10.2 <============== here
drwxr-xr-x 12 root root 4096 Jan 19 20:37 cuda-10.2
drwxr-xr-x  2 root root 4096 Apr 26  2018 etc
drwxr-xr-x  2 root root 4096 Apr 26  2018 games
drwxr-xr-x  2 root root 4096 Apr 26  2018 include
drwxr-xr-x  4 root root 4096 Jun  4  2018 lib
lrwxrwxrwx  1 root root    9 Apr 26  2018 man -> share/man
drwxr-xr-x  2 root root 4096 Apr 26  2018 sbin
drwxr-xr-x  7 root root 4096 May 21  2018 share
drwxr-xr-x  2 root root 4096 Apr 26  2018 src


# /usr/local/cuda/bin/nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Wed_Oct_23_21:14:42_PDT_2019
Cuda compilation tools, release 10.2, V10.2.89

# export PATH=${PATH}:/usr/local/cuda/bin

# nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Wed_Oct_23_21:14:42_PDT_2019
Cuda compilation tools, release 10.2, V10.2.89
PINTOPINTO
# apt install -y libhdf5-dev libc-ares-dev libeigen3-dev gcc \
                 gfortran libgfortran5 python3-pip \
                 libatlas3-base libatlas-base-dev \
                 libopenblas-dev libopenblas-base libblas-dev \
                 liblapack-dev cython libatlas-base-dev \
                 openmpi-bin libopenmpi-dev python3-dev \
                 wget curl make cmake zip unzip openjdk-8-jdk
# pip3 install gdown
PINTOPINTO

QEMUのstaticモードは下記のものを調達してうまく動作した。

$ sudo wget https://github.com/multiarch/qemu-user-static/releases/download/v5.1.0-8/qemu-aarch64-static -O /usr/bin/qemu-aarch64-static
$ sudo wget https://github.com/multiarch/qemu-user-static/releases/download/v5.1.0-8/qemu-arm-static -O /usr/bin/qemu-arm-static
PINTOPINTO

PyTorchとTensorFlowのインストール

$ apt update && apt-get install -y \
        automake autoconf libpng-dev nano wget npm \
        curl zip unzip libtool swig zlib1g-dev pkg-config \
        git wget xz-utils python3-mock libpython3-dev \
        libpython3-all-dev python3-pip g++ gcc make protobuf-compiler \
        pciutils cpio gosu git liblapack-dev liblapacke-dev

$ pip3 install --upgrade pip \
    && git clone -b v1.8.1 https://github.com/onnx/onnx.git \
    && cd onnx \
    && git submodule update --init --recursive \
    && sed -i -e "s/numpy>=1.16.6/numpy<=1.19.0/g" setup.py \
    && python3 setup.py install \
    && cd ..
    && pip3 install onnxruntime \
    && pip3 install gdown \
    && pip3 install cmake \
    && pip3 install ninja \
    && pip3 install yapf \
    && pip3 install six \
    && pip3 install wheel \
    && pip3 install moc \
    && wget https://nvidia.box.com/shared/static/cs3xn3td6sfgtene6jdvsxlr366m2dhq.whl \
       -O torch-1.7.0-cp36-cp36m-linux_aarch64.whl \
    && pip3 install torch-1.7.0-cp36-cp36m-linux_aarch64.whl \
    && rm torch-1.7.0-cp36-cp36m-linux_aarch64.whl \
    && wget https://developer.download.nvidia.com/compute/redist/jp/v45/tensorflow/tensorflow-2.3.1+nv20.12-cp36-cp36m-linux_aarch64.whl \
    && pip3 install tensorflow-2.3.1+nv20.12-cp36-cp36m-linux_aarch64.whl \
    && rm tensorflow-2.3.1+nv20.12-cp36-cp36m-linux_aarch64.whl \
    && ldconfig
PINTOPINTO
/usr/share/lintian/overrides
/usr/local/cuda
/usr/local/cuda-10.2
/usr/include
/usr/lib/pkgconfig
/usr/lib/aarch64-linux-gnu
/usr/src
/etc/alternatives
PINTOPINTO
$ docker run --rm --privileged multiarch/qemu-user-static --reset -p yes

$ docker pull nvcr.io/nvidia/l4t-base:r32.5.0
$ docker run --rm -it nvcr.io/nvidia/l4t-base:r32.5.0

$ apt update \
  && apt upgrade -y \
  && apt install -y \
   automake autoconf libpng-dev nano wget npm openjdk-8-jdk \
   curl zip unzip libtool swig zlib1g-dev pkg-config libprotoc-dev \
   git wget xz-utils python3-mock libpython3-dev libhdf5-dev \
   libpython3-all-dev python3-pip g++ gcc make protobuf-compiler \
   pciutils cpio gosu git liblapack-dev liblapacke-dev sox libsox-dev \
  && pip3 install --upgrade pip \
  && git clone -b v1.8.1 https://github.com/onnx/onnx.git \
  && cd onnx \
  && git submodule update --init --recursive \
  && sed -i -e "s/numpy>=1.16.6/numpy<=1.19.0/g" setup.py \
  && python3 setup.py install \
  && cd .. && rm -rf onnx \
  && pip3 install gdown \
  && pip3 install cmake \
  && pip3 install ninja \
  && pip3 install yapf \
  && pip3 install six \
  && pip3 install wheel \
  && pip3 install moc \
  && pip3 install cython \
  && pip3 install keras_applications==1.0.8 --no-deps \
  && pip3 install keras_preprocessing==1.1.0 --no-deps \
  && pip3 install numpy==1.19.0 \
  && pip3 install h5py==2.9.0 \
  && pip3 install pybind11

$ docker ps -a
CONTAINER ID   IMAGE                             COMMAND       CREATED          STATUS          PORTS     NAMES
10bdaeb86518   nvcr.io/nvidia/l4t-base:r32.5.0   "/bin/bash"   12 minutes ago   Up 12 minutes             great_williams

$ docker commit great_williams pinto0309/l4t-base:r32.5.0
$ docker push pinto0309/l4t-base:r32.5.0

$ docker run --rm -it pinto0309/l4t-base:r32.5.0

$ gdown --id 1yCTPM2aAoc0N4_sFIziPFEOr6c7so0bS
$ tar -zxvf usr.tar.gz -C / > /dev/null
$ rm usr.tar.gz
$ gdown --id 1uOZ_57GXBI05C6pITLLbMK4O2AcPw1jN
$ mv nv_tegra_release /etc/
$ export PATH=${PATH}:/usr/local/cuda/bin

$ gdown --id 1Vz5cWhNtGMYL8qHFJ5jgWUYVG-QNOJVY
$ chmod +x bazel
$ mv ./bazel /usr/local/bin

$ docker run --rm -it pinto0309/l4t-base:r32.5.0

$ git clone -b v2.4.1 https://github.com/tensorflow/tensorflow.git
$ cd tensorflow
$ ./configure

You have bazel 3.1.0- (@non-git) installed.
Please specify the location of python. [Default is /usr/bin/python3]: 


Found possible Python library paths:
  /usr/local/lib/python3.6/dist-packages
  /usr/lib/python3/dist-packages
Please input the desired Python library path to use.  Default is [/usr/local/lib/python3.6/dist-packages]

Do you wish to build TensorFlow with ROCm support? [y/N]: n
No ROCm support will be enabled for TensorFlow.

Do you wish to build TensorFlow with CUDA support? [y/N]: y
CUDA support will be enabled for TensorFlow.

Do you wish to build TensorFlow with TensorRT support? [y/N]: n
No TensorRT support will be enabled for TensorFlow.

Found CUDA 10.2 in:
    /usr/local/cuda-10.2/targets/aarch64-linux/lib
    /usr/local/cuda-10.2/targets/aarch64-linux/include
Found cuDNN 8 in:
    /usr/lib/aarch64-linux-gnu
    /usr/include


Please specify a list of comma-separated CUDA compute capabilities you want to build with.
You can find the compute capability of your device at: https://developer.nvidia.com/cuda-gpus. Each capability can be specified as "x.y" or "compute_xy" to include both virtual and binary GPU code, or as "sm_xy" to only include the binary code.
Please note that each additional compute capability significantly increases your build time and binary size, and that TensorFlow only supports compute capabilities >= 3.5 [Default is: 3.5,7.0]: 


Do you want to use clang as CUDA compiler? [y/N]: n
nvcc will be used as CUDA compiler.

Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]: 


Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -Wno-sign-compare]: 


Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]: 
Not configuring the WORKSPACE for Android builds.

Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
	--config=mkl         	# Build with MKL support.
	--config=mkl_aarch64 	# Build with oneDNN support for Aarch64.
	--config=monolithic  	# Config for mostly static monolithic build.
	--config=ngraph      	# Build with Intel nGraph support.
	--config=numa        	# Build with NUMA support.
	--config=dynamic_kernels	# (Experimental) Build kernels into separate shared objects.
	--config=v2          	# Build TensorFlow 2.x instead of 1.x.
Preconfigured Bazel build configs to DISABLE default on features:
	--config=noaws       	# Disable AWS S3 filesystem support.
	--config=nogcp       	# Disable GCP support.
	--config=nohdfs      	# Disable HDFS support.
	--config=nonccl      	# Disable NVIDIA NCCL support.
Configuration finished

$ bazel build \
--config=cuda \
--config=noaws \
--config=nohdfs \
--config=nonccl \
--config=v2 \
#--local_ram_resources=16384 \
#--local_cpu_resources=6 \
//tensorflow/tools/pip_package:build_pip_package
このスクラップは2021/02/26にクローズされました