five

jasmine_seed_0_nrows_2000_nclasses_10_ncols_100_stratify_True

收藏
OpenML2022-11-17 更新2024-05-23 收录
下载链接:
https://www.openml.org/search?type=data&sort=runs&status=active&id=44523
下载链接
链接失效反馈
官方服务:
资源简介:
Subsampling of the dataset jasmine (41143) with seed=0 args.nrows=2000 args.ncols=100 args.nclasses=10 args.no_stratify=True Generated with the following source code: ```python def subsample( self, seed: int, nrows_max: int = 2_000, ncols_max: int = 100, nclasses_max: int = 10, stratified: bool = True, ) -> Dataset: rng = np.random.default_rng(seed) x = self.x y = self.y # Uniformly sample classes = y.unique() if len(classes) > nclasses_max: vcs = y.value_counts() selected_classes = rng.choice( classes, size=nclasses_max, replace=False, p=vcs / sum(vcs), ) # Select the indices where one of these classes is present idxs = y.index[y.isin(classes)] x = x.iloc[idxs] y = y.iloc[idxs] # Uniformly sample columns if required if len(x.columns) > ncols_max: columns_idxs = rng.choice( list(range(len(x.columns))), size=ncols_max, replace=False ) sorted_column_idxs = sorted(columns_idxs) selected_columns = list(x.columns[sorted_column_idxs]) x = x[selected_columns] else: sorted_column_idxs = list(range(len(x.columns))) if len(x) > nrows_max: # Stratify accordingly target_name = y.name data = pd.concat((x, y), axis="columns") _, subset = train_test_split( data, test_size=nrows_max, stratify=data[target_name], shuffle=True, random_state=seed, ) x = subset.drop(target_name, axis="columns") y = subset[target_name] # We need to convert categorical columns to string for openml categorical_mask = [self.categorical_mask[i] for i in sorted_column_idxs] columns = list(x.columns) return Dataset( # Technically this is not the same but it's where it was derived from dataset=self.dataset, x=x, y=y, categorical_mask=categorical_mask, columns=columns, ) ```

对数据集jasmine(41143)执行下采样操作,所用参数如下: 随机种子(seed)=0, 最大允许采样行数(args.nrows)=2000, 最大允许采样列数(args.ncols)=100, 最大允许采样类别数(args.nclasses)=10, 未启用分层采样(args.no_stratify=True)。 本次下采样通过以下源代码生成: python def subsample( self, seed: int, nrows_max: int = 2_000, ncols_max: int = 100, nclasses_max: int = 10, stratified: bool = True, ) -> 数据集(Dataset): # 初始化NumPy(np)随机数生成器 rng = np.random.default_rng(seed) # x为特征矩阵,y为标签向量 x = self.x y = self.y # 执行均匀采样 classes = y.unique() if len(classes) > nclasses_max: vcs = y.value_counts() selected_classes = rng.choice( classes, size=nclasses_max, replace=False, p=vcs / sum(vcs), ) # 选取属于上述选中类别的样本索引 idxs = y.index[y.isin(classes)] x = x.iloc[idxs] y = y.iloc[idxs] # 若需采样特征列,则执行均匀列采样 if len(x.columns) > ncols_max: columns_idxs = rng.choice( list(range(len(x.columns))), size=ncols_max, replace=False ) sorted_column_idxs = sorted(columns_idxs) selected_columns = list(x.columns[sorted_column_idxs]) x = x[selected_columns] else: sorted_column_idxs = list(range(len(x.columns))) if len(x) > nrows_max: # 据此执行分层采样 target_name = y.name data = pd.concat((x, y), axis="columns") _, subset = train_test_split( data, test_size=nrows_max, stratify=data[target_name], shuffle=True, random_state=seed, ) x = subset.drop(target_name, axis="columns") y = subset[target_name] # 为适配OpenML平台,需将类别型特征列转换为字符串类型 categorical_mask = [self.categorical_mask[i] for i in sorted_column_idxs] columns = list(x.columns) return Dataset( # 严格来说,该数据集与原数据集并不完全一致,但二者的衍生关系如上 dataset=self.dataset, x=x, y=y, categorical_mask=categorical_mask, columns=columns, )
创建时间:
2022-11-17
二维码
社区交流群
二维码
科研交流群
商业服务