Datasets
Simulation designs from Section 4 of Tanaka and Matsui (2023).
SimulatedData
dataclass
One draw of (X, y) together with the known active set.
example1_sigma(p, phi)
Covariance of Example 1.
Sigma_jj = 1, Sigma_jk = phi off-diagonal except row/column 4
(1-based), which equals sqrt(phi).
example2_sigma(p, phi)
Example 1 covariance with variable 5 uncorrelated with the rest.
make_example1(n=100, p=1000, phi=0.5, *, seed=None)
Example 1, with the coefficient that makes Cov(y, x_4) = 0.
The printed regression writes -15 x_4. Fan and Lv (2008), Example II,
and Zhao et al. (2020), Example 1, which this design follows, use
-15 sqrt(phi) x_4. With the printed covariance that is the unique
coefficient that makes column 4 uncorrelated with y, which is the
mechanism the paper describes. The typeset -15 leaves a correlation
of 15 sqrt(phi) - 15.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n
|
int
|
Sample size, dimension, and equicorrelation. The paper uses
|
100
|
p
|
int
|
Sample size, dimension, and equicorrelation. The paper uses
|
100
|
phi
|
int
|
Sample size, dimension, and equicorrelation. The paper uses
|
100
|
seed
|
int | Generator | None
|
|
None
|
make_example2(n=100, p=1000, phi=0.5, *, seed=None)
Example 2: Example 1 plus an uncorrelated fifth active variable.
make_example3(n=100, p=1000, phi=0.5, *, seed=None)
Example 3: Example 2 plus x_6 = 0.8 x_5 + delta.
Variable 6 is inactive. delta ~ N(0, 0.01).
make_example4(n=100, p=1000, *, d=3, m=None, seed=None)
Example 4: the spike model of specification Section 6.4.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n
|
int
|
Sample size, dimension, and number of large factors (paper: |
100
|
p
|
int
|
Sample size, dimension, and number of large factors (paper: |
100
|
d
|
int
|
Sample size, dimension, and number of large factors (paper: |
100
|
m
|
int | None
|
Number of medium factors. The paper uses |
None
|
seed
|
int | Generator | None
|
|
None
|
min_eigenvalue(sigma)
Smallest eigenvalue, used as the Section 6.5 PSD gate.
Fetch scripts for the two real datasets in Section 5 of the paper.
Neither dataset is redistributed. These helpers download from the public sources the paper cites, cache locally, and raise a clear error if a source is unavailable.
fetch_hydraulic(cache_dir=None, *, downloader=None)
Download the UCI hydraulic-systems archive into cache_dir.
Returns the path to the zip. Parsing into the paper's (X, y) layout is
left to reproduction/run_real_data.py, because the 17-sensor expansion
is study-specific.
fetch_sp500_info()
Return the documented S&P 500 source URLs. No automatic download.
The Kaggle source requires an account, so this function only documents
the locations. reproduction/run_real_data.py reads a user-supplied
directory of CSVs.
verify_checksum(path, expected)
Compare a file to a published SHA-256 hex digest.