tfnn/HeadsNet

---
license: mit
tags:
- thispersondoesnotexist
- stylegan
- stylegan2
- mesh
- model
- 3d
- asset
- generative
- triposr
pretty_name: HeadsNet
size_categories:
- 1K<n<10K
---

# HeadsNet

The basic concept is to train a FNN/MLP on vertex data of 3D heads so that it can then re-produce random 3D heads.

This dataset uses the [thispersondoesnotexist_to_triposr_6748_3D_Heads](https://huggingface.co/datasets/tfnn/thispersondoesnotexist_to_triposr_6748_3D_Heads) dataset as a foundation.

The heads dataset was collecting using the scraper [Dataset_Scraper.7z](https://huggingface.co/datasets/tfnn/HeadsNet/resolve/main/Dataset_Scraper.7z?download=true) based on [TripoSR](https://github.com/VAST-AI-Research/TripoSR) which converts the 2D images from [ThisPersonDoesNotExist](https://thispersondoesnotexist.com/) into 3D meshes. _(using [this marching cubes improvement](https://github.com/VAST-AI-Research/TripoSR/issues/22#issuecomment-2010318709) by [thatname/zephyr](https://github.com/thatname))_

Vertex Normals need to be generated before we can work with this dataset, the easiest method to achieve this is with a simple [Blender](https://www.blender.org/) script:
```
import bpy
import glob
import pathlib
from os import mkdir
from os.path import isdir
importDir = "ply/"
outputDir = "ply_norm/"
if not isdir(outputDir): mkdir(outputDir)

for file in glob.glob(importDir + "*.ply"):
model_name = pathlib.Path(file).stem
if pathlib.Path(outputDir+model_name+'.ply').is_file() == True: continue
bpy.ops.wm.ply_import(filepath=file)
bpy.ops.wm.ply_export(
filepath=outputDir+model_name+'.ply',
filter_glob='*.ply',
check_existing=False,
ascii_format=False,
export_selected_objects=False,
apply_modifiers=True,
export_triangulated_mesh=True,
export_normals=True,
export_uv=False,
export_colors='SRGB',
global_scale=1.0,
forward_axis='Y',
up_axis='Z'
)
bpy.ops.object.select_all(action='SELECT')
bpy.ops.object.delete(use_global=False)
bpy.ops.outliner.orphans_purge()
bpy.ops.outliner.orphans_purge()
bpy.ops.outliner.orphans_purge()
```
_Importing the PLY without normals causes Blender to automatically generate them._

At this point the PLY files now need to be converted to training data, for this I wrote a C program [DatasetGen_2_6.7z](https://huggingface.co/datasets/tfnn/HeadsNet/resolve/main/DatasetGen_2_6.7z?download=true) using [RPLY](https://w3.impa.br/~diego/software/rply/) to load the PLY files and convert them to binary data which I have provided here [HeadsNet-2-6.7z](https://huggingface.co/datasets/tfnn/HeadsNet/resolve/main/HeadsNet-2-6.7z?download=true).

It's always good to [NaN](https://en.wikipedia.org/wiki/NaN) check your training data after generating it so I have provided a simple Python script for that here [nan_check.py](https://huggingface.co/datasets/tfnn/HeadsNet/resolve/main/nan_check.py?download=true).

This binary training data can be loaded into Python using [Numpy](https://numpy.org/):
```
load_x = []
with open("train_x.dat", 'rb') as f:
load_x = np.fromfile(f, dtype=np.float32)

load_y = []
with open("train_y.dat", 'rb') as f:
load_y = np.fromfile(f, dtype=np.float32)
```

The data can then be reshaped and saved back out as a numpy array which makes for faster loading:
```
inputsize = 2
outputsize = 6
training_samples = 632847695
train_x = np.reshape(load_x, [training_samples, inputsize])
train_y = np.reshape(load_y, [training_samples, outputsize])
np.save("train_x.npy", train_x)
np.save("train_y.npy", train_y)
```
_632,847,695 samples, each sample is 2 components for train_x (random seed & 0-1 unit sphere position index) and 6 components for train_y (vertex position [x,y,z] & vertex color [r,g,b])._

The basic premise of how this network is trained and thus how the dataset is generated in the C program is:
1. All models are pre-scaled to a normal cubic scale and then scaled again by 0.55 so that they all fit within a unit sphere.
2. All model vertices are reverse traced from the vertex position to the perimeter of the unit sphere using the vertex normal as the direction.
3. The nearest position on a 10,242 vertex icosphere is found and the network is trained to output the model vertex position and vertex color (6 components) at the index of the icosphere vertex.
4. The icosphere vertex index is scaled to a 0-1 range before being input to the network.
5. The network only has two input parameters, the other parameter is a 0-1 model ID which is randomly selected and all vertices for a specific model are trained into the network using the randomly selected ID. This ID does not change per-vertex it only changes per 3D model.
6. The ID allows the user to use this parameter as a sort of hyper-parameter for the random seed: to generate a random Head using this network you would input a random 0-1 seed and then iterate the icosphere index parameter to some sample range between 0-1 so if you wanted a 20,000 vertex head you would iterate between 0-1 at 20,000 increments of 0.00005 as the network outputs one vertex position and vertex color for each forward-pass.

* 1st input parameter = random seed
* 2nd input parameter = icosphere index

More about this type of network topology can be read here: https://gist.github.com/mrbid/1eacdd9d9239b2d324a3fa88591ff852

## Improvements
* Future networks will have 3 additional input parameters one for each x,y,z of a unit vector for the ray direction from the icosphere index.
* The unit vector used to train the network will just be the vertex normal from the 3D model but inverted.
* When performing inference more forward-passes would need to be performed as some density of rays in a 30° or similar cone angle pointing to 0,0,0 would need to be performed per icosphere index position.
* This could result in higher quality outputs; at the cost of an order of magnitude more forward-pass iterations.

## Updates
* A new dataset has been generated [HeadsNet-2-6_v2.7z](https://huggingface.co/datasets/tfnn/HeadsNet/resolve/main/HeadsNet-2-6_v2.7z?download=true), the old one uses a 10,242 vertex unit icosphere and the new one uses a 655,362 vertex unit icosphere, this should lead to a higher quality network. Start training with it instantly using [HeadsNet_v2_Trainer_with_Dataset.7z](https://huggingface.co/datasets/tfnn/HeadsNet/resolve/main/HeadsNet_v2_Trainer_with_Dataset.7z?download=true).
* The system didn't work out, here I have trained models of various qualities: [HeadsNet_Trained_Models.7z](https://huggingface.co/datasets/tfnn/HeadsNet/resolve/main/HeadsNet_Trained_Models.7z?download=true). The network has some potential, with a better refined dataset and better network topology it could prove more successful.
* Added [HeadsNet3](https://huggingface.co/datasets/tfnn/HeadsNet/resolve/main/HeadsNet3.7z?download=true) using the [FaceTo3D](https://huggingface.co/datasets/tfnn/FaceTo3D) dataset a pivot where I attempted to train an FNN/MLP on a 1024 component input vector of a 32x32 grayscale image of a face to output a 32x32x32 grayscale voxel volume of a 3D head. Results where not overwhelmingly positive. Had higher hopes.
* Added [HeadsNet4](https://huggingface.co/datasets/tfnn/HeadsNet/resolve/main/HeadsNet4.7z?download=true) this version uses 32^3 micro MLP's with a single voxel grayscale output per network, the trained datasetd are included and the program to generate them but you will need to download the PLY files to generate the dataset from the [FaceTo3D](https://huggingface.co/datasets/tfnn/FaceTo3D) repository.
* This is being continued with more success over at GitHub: https://github.com/mrbid/FaceTo3D

原始信息汇总

数据集概述：HeadsNet

数据集基本信息

• 名称：HeadsNet
• 许可证：MIT
• 标签：thispersondoesnotexist, stylegan, stylegan2, mesh, model, 3d, asset, generative, triposr
• 美观名称：HeadsNet
• 大小范围：1K<n<10K

数据集内容与结构

• 基础数据集：使用thispersondoesnotexist_to_triposr_6748_3D_Heads作为基础数据集。
• 数据收集：通过TripoSR将ThisPersonDoesNotExist的2D图像转换为3D网格。
• 数据处理：使用Blender脚本生成顶点法线，并通过C程序DatasetGen_2_6.7z将PLY文件转换为二进制数据。
• 数据检查：提供nan_check.py脚本用于检查训练数据中的NaN值。

数据集使用

• 数据加载：使用Numpy加载二进制训练数据，并重新塑形为Numpy数组以加速加载。
• 训练方法：网络训练基于随机种子和icosphere索引，每个3D模型使用随机选择的ID进行训练。

数据集更新与改进

• 新数据集：HeadsNet-2-6_v2.7z使用655,362顶点的单位icosphere，提高网络质量。
• 训练模型：提供HeadsNet_Trained_Models.7z包含不同质量的训练模型。
• 新版本：HeadsNet3和HeadsNet4尝试使用不同的数据集和网络结构。

数据集未来改进

• 增加输入参数：未来网络将增加三个输入参数，用于表示从icosphere索引发出的射线方向的单位向量。
• 提高输出质量：通过增加前向传递的迭代次数，可以在保持相同输出质量的同时，提高输出的密度。