ja-topic-graph-qa

# Japanese Chart Visual Question Answering Dataset ## Dataset Description This dataset contains Japanese chart/graph visualizations with corresponding question-answer pairs for visual question answering (VQA) tasks. The dataset is designed for training and evaluating multimodal language models on Japanese chart understanding tasks. ### Dataset Summary This dataset consists of automatically generated chart visualizations (bar charts, line charts, pie charts, histograms) with Japanese labels, paired with Japanese question-answer pairs. ### Supported Tasks - **Visual Question Answering (VQA)**: Answer questions about chart content - **Chart Understanding**: Understand and interpret chart visualizations - **Multimodal Dialogue**: Multi-turn conversations about charts - **Data Analysis**: Extract and reason about data from visualizations ### Languages The dataset is primarily in **Japanese (ja)**, with all questions, answers, and chart labels in Japanese. ## Dataset Structure ### Data Instances Each instance contains: - **Image**: A chart visualization (PNG format, high resolution) - **Conversations**: Multi-turn dialogue format with question-answer pairs - **Metadata**: Topic, chart type, persona, source data, and generation code ### Data Fields #### Core Fields (for training) - `image`: Chart image (PIL Image format) - `conversations`: Conversation format with question and answer pairs in multi-turn dialogue ```python [ {"from": "human", "value": "問題1"}, {"from": "gpt", "value": "答え1"}, {"from": "human", "value": "問題2"}, {"from": "gpt", "value": "答え2"}, ... ] ``` - `source`: Data source identifier, fixed value `"ja-topic-graph-qa"` #### Metadata Fields - `index`: Original entry index - `topic`: Chart topic in Japanese (e.g., "仙台市における年代別の飲料購入頻度の比較") - `figure_type`: Type of chart (bar chart, line chart, pie chart, histogram) - `persona`: Persona description used for generation - `data`: CSV data used to generate the chart - `code`: Python code (Matplotlib) used to generate the chart - `questions`: List of all questions in Japanese - `answers`: List of all answers in Japanese - `explanations`: List of all explanations in Japanese - `num_qa_pairs`: Number of question-answer pairs for this chart - `image_path`: Original image path ### Data Splits The dataset can be split into train/test sets. If split, the default ratio is typically 90/10. ## Dataset Statistics - **Total Images**: 4,557 charts (successfully generated) - **Total QA Pairs**: 35,702 question-answer pairs - **Average QA Pairs per Image**: ~7.8 pairs - **Chart Types**: Bar charts (25.4%), line charts (24.3%), pie charts (24.8%), histograms (25.5%) ## Dataset Creation ### Source Data The dataset is automatically generated using: - **Chart Generation**: Python (Matplotlib) code generation with LLM - **Data Generation**: CSV data creation with realistic Japanese topics - **QA Generation**: Qwen3-VL model for generating Japanese question-answer pairs ### Annotations - **Annotation process**: Automated using Qwen3-VL vision-language model - **Annotation quality**: High-quality Japanese Q&A pairs with explanations - **Annotation diversity**: Multiple question types and complexity levels ### Personal and Sensitive Information The dataset contains synthetic data generated for research purposes. No real personal information is included. ## Considerations for Using the Data ### Social Impact of Dataset This dataset can be used to: - Improve Japanese multimodal AI capabilities - Enable better chart understanding in Japanese contexts - Support data analysis and visualization tools for Japanese users ### Discussion of Biases - The dataset focuses on Japanese language and cultural contexts - Chart topics are diverse but may not cover all domains - Generated data may reflect biases in the generation models ### Other Known Limitations - Charts are programmatically generated, not real-world data - Question complexity is controlled but may not match all real-world scenarios - Limited to specific chart types (bar, line, pie, histogram) ## Getting Started with the Dataset ### Loading the Dataset ```python from datasets import load_dataset # Load the dataset dataset = load_dataset("your-username/japanese-chart-vqa-dataset") # Or load from local directory from datasets import load_from_disk dataset = load_from_disk("hf_dataset") ``` ### Basic Usage ```python # Access a sample sample = dataset['train'][0] # Get the image image = sample['image'] image.show() # Display the chart # Get conversations (multi-turn dialogue) conversations = sample['conversations'] for turn in conversations: print(f"{turn['from']}: {turn['value']}") # Get metadata print(f"Topic: {sample['topic']}") print(f"Chart Type: {sample['figure_type']}") print(f"Number of QA pairs: {sample['num_qa_pairs']}") # Get all questions and answers questions = sample['questions'] answers = sample['answers'] for q, a in zip(questions, answers): print(f"Q: {q}") print(f"A: {a}\n") ``` ### Training Example ```python from datasets import load_dataset from transformers import AutoProcessor, AutoModelForVision2Seq # Load dataset dataset = load_dataset("your-username/japanese-chart-vqa-dataset") # Load model and processor processor = AutoProcessor.from_pretrained("model-name") model = AutoModelForVision2Seq.from_pretrained("model-name") # Prepare data for training def prepare_example(example): return { "image": example["image"], "conversations": example["conversations"] } train_dataset = dataset["train"].map(prepare_example) ``` ## Citation ## License ## Dataset Card Contact ## Acknowledgments This dataset was generated using: - gpt-4o-2024-11-20 for topic/data generation - gpt-4o-2024-11-20 for code generation - gpt-4o-2024-11-20 for QA generation - Matplotlib for chart/graph generation