Submission instructions

Four quality badges can be considered for an artifact. Along with submitting the final version (camera ready) of the article, authors can submit the artifact(s) related to the article. For artifacts to be judged, it is mandatory to submit an appendix describing which badges should be considered in the evaluation of the work artifact(s) and providing the necessary instructions for reviewers to be able to properly deal with them. (especially in the case of software or data).

Requirements

For the work/artifact to be eligible to receive a badge, some requirements must be met, as described below.

Available Artifacts (SeloD)

Code and/or data is expected to be available in a stable repository (such as GitHub or GitLab). In this repository, a README.md is expected with minimal documentation (but clear enough for understanding), describing the objective of the artifact(s) (s), with the respective title and summary of the article.

Functional Artifacts (SeloF)

It is expected that the program code(s) of the artifact(s) can be executed and reviewers can observe its functionalities. To obtain this badge, it is important that additional information is present in the repository's README.md, such as:

  1. list of dependencies;
  2. list of dependencies/languages/environment versions;
  3. description of the execution environment;
  4. installation and execution instructions;
  5. one or more execution examples.

Sustainable Artifacts (SeloS)

It is expected that the program code(s) of the artifact(s) are modularized, organized, intelligible and easy to understand **. To obtain the badge you must:

  1. there is documentation of the code(s) (describing files, functions, APIs, etc.);
  2. there is minimum readability in the code(s) and other artifacts;
  3. it is possible for reviewers to identify the main claims of the article in the artifact(s).

Reproducible Experiments (SeloR)

It is expected that the reviewer will be able to reproduce the main claims presented in the article. To obtain this badge you must have:

  1. instruction to execute the code(s) in order to reproduce and confirm the main claims of the article (e.g., results of the main graphs/tables);
  2. description of the process of carrying out the experiments to reach the result(s) of the article;
  3. description of specific technical details of the environment (when applicable) such as, for example, details of the infrastructure used in Amazon Cloud or Google Cloud. Eventually, include access keys and other information that allows the experiment to be reproduced.

Appendix (required)

To facilitate the artifact evaluation process, an appendix template (required) was created where the authors describe which badges should be considered in the artifact evaluation process and include information to assist reviewers of the Artifacts Technical Committee. The appendix LaTeX template is available at Example-Appendix.

It is mandatory that Sections 1 to 4 are present. If you have any questions, please contact us.

Note: Before submitting their artifact (appendix), it is strongly recommended that authors install and run their artifact in a new environment (virtual machine) following only the instructions in the appendix to confirm that the instructions are consistent and complete.

Note: Remember that the entire CTA review process takes into account the information in the Appendix.

Examples of SBSeg'23 Artifacts

An example from Appendix. Result after the committee evaluation process Article.