[3주차] 건물 3D화 모델 찾기

 

<GitHub- 3D building reconstruction>

GitHub - chrise96/3D_building_reconstruction: MSc Computer Science project. Automatically enhance CityGML LOD2 buildings with facade details, by using a panoramic image sequence and building footprint data. NOTE: Amsterdam Panorama API is currently offline.

MSc Computer Science project. Automatically enhance CityGML LOD2 buildings with facade details, by using a panoramic image sequence and building footprint data. NOTE: Amsterdam Panorama API is curr..

https://github.com/chrise96/3D_building_reconstruction

<GitHub- 3D building reconstruction>

GitHub - jshmsjh/3d-building-reconstruction

Contribute to jshmsjh/3d-building-reconstruction development by creating an account on GitHub.

https://github.com/jshmsjh/3d-building-reconstruction

 

<Papers with code - PolyGNN 23년 7월 17일 따끈 신상(코드는 없음)>

• Poly 형태라 우리 task와 맞지 않음

 

https://paperswithcode.com/paper/polygnn-polyhedron-based-graph-neural-network

GitHub - chenzhaiyu/points2poly: Reconstructing compact building models from point clouds using deep implicit fields [ISPRS 2022]

Reconstructing compact building models from point clouds using deep implicit fields [ISPRS 2022] - GitHub - chenzhaiyu/points2poly: Reconstructing compact building models from point clouds using de...

https://github.com/chenzhaiyu/points2poly

 

<위성 사진 활용한 3D building reconstruction - code x>

3D building reconstruction from single street view images using deep learning

3D building models are an established instance of geospatial information in the built environment, but their acquisition remains complex and topical. …

https://www.sciencedirect.com/science/article/pii/S1569843222000619

Automatic 3D building reconstruction from multi-view aerial images with deep learning

The study presented in this paper introduced a new fully automatic three-dimensional building reconstruction method that can generate first level of d…

https://www.sciencedirect.com/science/article/pii/S092427162030318X

---

 

채원이와 다른점 ⇒ 데이터(학습 데이터 위주) ⇒ 건물 하나하나에 대한 정확성 ⇒ 구글 어스에 심는다는 가정

💡

3주차 과제: 건물 3D화 모델 찾기 → 구현 or 대안점

❓

<3D buildings reconstruction에 있어서 가장 중요한 point> 1. 데이터셋의 유무 2. 건물에 대한 높이와 거리 고려 3. 3D화 4. Texture의 적용

📝

<방법>

1. 다양한 각도에서 찍은 건물 사진(없어요 그냥) + 위성사진 ⇒ 2d to 3d point cloud화 (AE) or

2. Lidar dataset -> 물체와의 거리 측정 및 3d point cloud 생성

3. 3d point cloud to mesh(3D 생성)

4. 3d 건물 표면 적용 (mesh/texture or gan)

5. 전체적인 건물끼리의 거리 표현 (위성을 활용한) (구글 어스 활용)

---

 

1. DATASETS

1. 2D Image to 3D Point Cloud

<Learning Efficient Point Cloud Generation for Dense 3D Object Reconstruction> (2017)

📚 https://medium.com/vitalify-asia/create-3d-model-from-a-single-2d-image-in-pytorch-917aca00bb07 (간단 리뷰)

📚 https://arxiv.org/pdf/1706.07036.pdf

📚 https://arxiv.org/abs/1706.07036

💻 https://github.com/lkhphuc/pytorch-3d-point-cloud-generation (pytorch)

- Auto Encoder 활용 - 하나의 RGB 이미지에서 3D point cloud화 - 3D 좌표 (x,y,z) ⇒ 건물의 높이까지 파악 가능

 

 

2. 위성 데이터 기반 Lidar Datasets(Light Detection and Ranging)

<Extract 3D buildings from lidar data>

- Lidar는 레이저를 사용하여 물체와의 거리를 측정하여 3D 포인트 클라우드를 생성하는 기술로, 건물의 3D 형상을 정확하게 캡처함 - 요구사항: GIS 활용 - Point Cloud까지만 추출 (LAS Point Cloud)

📚 https://learn.arcgis.com/en/projects/extract-3d-buildings-from-lidar-data/

📚 https://arxiv.org/pdf/1706.07036.pdf

📚 https://arxiv.org/abs/1706.07036

💻 https://github.com/lkhphuc/pytorch-3d-point-cloud-generation (pytorch)

2. 3D reconstruction (Points to Mesh)

1. Point2Mesh: A Self-Prior for Deformable Meshes (2020)

📚 https://arxiv.org/abs/2005.11084 💻 https://github.com/ranahanocka/point2mesh

point를 mesh 형태로 생성 후, 이미지를 입히는 형식으로 가고자 함

 

2. Points2Poly:Reconstructing Compact Building Models from Point Clouds Using Deep Implicit Fields (2021)

📚 https://arxiv.org/pdf/2112.13142v3.pdf 💻 https://github.com/chenzhaiyu/points2poly (?)

3. Texture mapping

1. Polygon-based Texture Mapping for Cyber City 3D building Models

📚 http://gcl.csrsr.ncu.edu.tw/publication/bldgTexture.dist.pdf 💻 코드가.. 없어요……. ㅜㅜ

 

---

⇒ 생성쪽이어야 하나..? ㅜ

2. Fine Detailed Texture Learning for 3D Meshes with Generative Models

📚 https://arxiv.org/pdf/2203.09362.pdf

💻 코드 x

3. GET3D: A Generative Model of High Quality 3D Textured Shapes Learned from Images

Mesh to GAN

📚https://arxiv.org/pdf/2209.11163v1.pdf 💻 https://github.com/nv-tlabs/GET3D

https://nv-tlabs.github.io/GET3D/assets/text2mesh-house.mp4

4. Google 어스 활용

google 어스의 필요요건이 뭔지는 잘 모르겠지만 위성 기반으로 거리에 맞게 각 건물 위치에 3D object를 넣어주면 되지 않을까라는 저의 이론 끝.