Great! We are always on the lookout for more contributors to our code base.
Firstly, if you are unsure or afraid of anything, just ask or submit the issue or pull request anyways. You won’t be yelled at for giving your best effort. The worst that can happen is that you’ll be politely asked to change something. We appreciate any sort of contributions and don’t want a wall of rules to get in the way of that.
However, for those individuals who want a bit more guidance on the best way to contribute to the project, read on. This document will cover all the points we’re looking for in your contributions, raising your chances of quickly merging or addressing your contributions.
There are a few simple guidelines that you need to follow before providing your hacks.
Bug Reports and Feature Requests¶
If you encountered a problem when using NNI, or have an idea for a new feature, your feedbacks are always welcome. Here are some possible channels:
File an issue on GitHub.
Open or participate in a discussion.
Discuss on the NNI Gitter in NNI.
Join IM discussion groups:
Looking for an existing issue¶
Before you create a new issue, please do a search in open issues to see if the issue or feature request has already been filed.
Be sure to scan through the most popular feature requests.
If you find your issue already exists, make relevant comments and add your reaction. Use a reaction in place of a “+1” comment:
👍 - upvote
👎 - downvote
If you cannot find an existing issue that describes your bug or feature, create a new issue following the guidelines below.
Writing good bug reports or feature requests¶
File a single issue per problem and feature request. Do not enumerate multiple bugs or feature requests in the same issue.
Provide as much information as you think might relevant to the context (thinking the issue is assigning to you, what kinds of info you will need to debug it!!!). To give you a general idea about what kinds of info are useful for developers to dig out the issue, we had provided issue template for you.
Once you had submitted an issue, be sure to follow it for questions and discussions.
Once the bug is fixed or feature is addressed, be sure to close the issue.
There is always something more that is required, to make it easier to suit your use-cases. Before starting to write code, we recommend checking for issues on GitHub or open a new issue to initiate a discussion. There could be cases where people are already working on a fix, or similar features have already been under discussion.
To contribute code, you first need to find the NNI code repo located on GitHub. Firstly, fork the repository under your own GitHub handle. After cloning the repository, add, commit, push and squash (if necessary) the changes with detailed commit messages to your fork. From where you can proceed to making a pull request. The pull request will then be reviewed by our core maintainers before merging into master branch. Here is a step-by-step guide for this process.
Contributions to NNI should follow our code of conduct. Please see details here.
Find the code snippet that concerns you¶
The NNI repository is large code-base. High-level speaking, it can be decomposed into several core parts:
nni: the core Python package that contains most features of hyper-parameter tuner, neural architecture search, model compression.
nni_managerthat manages experiments and training services, and
test: unit test and integration test, alongside their configurations.
See Architecture Overview if you are interested in details.
Get started with development¶
NNI development environment supports Ubuntu 1604 (or above), and Windows 10 with Python 3.7+ (documentation build requires Python 3.8+). We recommend using conda on Windows.
Fork the NNI’s GitHub repository and clone the forked repository to your machine.
git clone https://github.com/<your_github_handle>/nni.git
Create a new working branch. Use any name you like.
cd nni git checkout -b feature-xyz
Install NNI from source code if you need to modify the source code, and test it.
python3 -m pip install -U -r dependencies/setup.txt python3 -m pip install -r dependencies/develop.txt python3 setup.py develop
This installs NNI in development mode, so you don’t need to reinstall it after edit.
Try to start an experiment to check if your environment is ready. For example, run the command
nnictl create --config examples/trials/mnist-pytorch/config.yml
And open WebUI to check if everything is OK. Or check the version of installed NNI,
>>> import nni >>> nni.__version__ '999.dev0'
Please don’t run test under the same folder where the NNI repository is located. As the repository is probably also called
nni, it could import the wrong
Write your code along with tests to verify whether the bug is fixed, or the feature works as expected.
Reload changes. For Python, nothing needs to be done, because the code is already linked to package folders. For TypeScript on Linux and MacOS,
ts/nni_manageris changed, run
yarn watchunder this folder. It will watch and build code continually. The
nnictlneed to be restarted to reload NNI manager.
ts/webuiis changed, run
yarn dev, which will run a mock API server and a webpack dev server simultaneously. Use
EXPERIMENTenvironment variable (e.g.,
mnist-tfv1-running) to specify the mock data being used. Built-in mock experiments are listed in
src/webui/mock. An example of the full command is
EXPERIMENT=mnist-tfv1-running yarn dev.
For TypeScript on Windows, currently you must rebuild TypeScript modules with python3 setup.py build_ts after edit.
Commit and push your changes, and submit your pull request!
We expect all contributors to respect the following coding styles and naming conventions upon their contribution.
We follow PEP8 for Python code and naming conventions, do try to adhere to the same when making a pull request. Our pull request has a mandatory code scan with
To scan your own code locally, run
python -m pylint --rcfile pylintrc nni
One can also take the help of auto-format tools such as autopep8, which will automatically resolve most of the styling issues.
We recommend documenting all the methods and classes in your code. Follow NumPy Docstring Style for Python Docstring Conventions.
For function docstring, description, Parameters, and Returns are mandatory.
For class docstring, description is mandatory. Optionally Parameters and Attributes. The parameters of
__init__should be documented in the docstring of class.
For docstring to describe
dict, which is commonly used in our hyper-parameter format description, please refer to Internal Guideline on Writing Standards.
Basically, you can use ReStructuredText syntax in docstrings, without some exceptions. For example, custom headings are not allowed in docstrings.
TypeScript code checks can be done with,
# for nni manager cd ts/nni_manager yarn eslint # for webui cd ts/webui yarn sanity-check
When a new feature is added or a bug is fixed, tests are highly recommended to make sure that the fix is effective or the feature won’t break in future. There are two types of tests in NNI:
Unit test (UT): each test targets at a specific class / function / module.
Integration test (IT): each test is an end-to-end example / demo.
Unit test (Python)¶
Python UT are located in
test/ut/ folder. We use pytest to launch the tests, and the working directory is
pytest can be used on a single file or a single test function.
pytest sdk/test_tuner.py pytest sdk/test_tuner.py::test_tpe
Unit test (TypeScript)¶
TypeScript UT are paired with TypeScript code. Use
yarn test to run them.
The integration tests can be found in
The integration tests are run on Azure DevOps platform on a daily basis, in order to make sure that our examples and training service integrations work properly. However, for critical changes that have impacts on the core functionalities of NNI, we recommend to trigger the pipeline on the pull request branch.
The integration tests won’t be automatically triggered on pull requests. You might need to contact the core developers to help you trigger the tests.
Build and check documentation¶
Our documentation is located under
docs/ folder. The following command can be used to build the documentation.
cd docs make en
If you experience issues in building documentation, and see errors like:
Could not import extension xxx (exception: No module named 'xxx'): please check your development environment and make sure dependencies have been properly installed: Get started with development.
unsupported pickle protocol: 5: please upgrade to Python 3.8.
autodoc: No module named 'xxx': some dependencies in
dependencies/are not installed. In this case, documentation can be still mostly successfully built, but some API reference could be missing.
It’s also highly recommended taking care of every WARNING during the build, which is very likely the signal of a deadlink and other annoying issues. Our code check will also make sure that the documentation build completes with no warning.
The built documentation can be found in
Always use your web browser to check the documentation before committing your change.
Live Server is a great extension if you are looking for a static-files server to serve contents in
Writing new documents¶
ReStructuredText is our documentation language. Please find the reference of RST here.
Sphinx has an excellent cheatsheet of rst which contains almost everything you might need to know to write a elegant document.
Dealing with sections.
= for sections.
- for subsections.
^ for subsubsections.
" for paragraphs.
Dealing with images. Images should be put into
docs/img folder. Then, reference the image in the document with relative links. For example,
.. image:: ../../img/example.png.
Dealing with codes. We recommend using
.. code-block:: python to start a code block. The
python here annotates the syntax highlighting.
Dealing with links. Use
:doc:`./relative/to/my_doc` for links to another doc (no suffix like
.rst). To reference a specific section, please use
:ref: (see Cross-referencing arbitrary locations). For general links that
:ref: can’t handle, you can also use
`Link text <https://domain.invalid/>`_ for inline web links. Note that use one underline might cause “duplicated target name” error when multiple targets share the same name. In that case, use double-underline to avoid the error:
`Link text <https://domain.invalid/>`__.
Other than built-in directives provided by Sphinx, we also provide some custom directives:
.. cardlinkitem::: A tutorial card, useful in Examples.
:githublink:`text <path/to/file.ext>`: reference a file on the GitHub. Linked to the same commit id as where the documentation is built.
Writing new tutorials¶
Our tutorials are powered by sphinx-gallery <https://sphinx-gallery.github.io/>. Sphinx-gallery is an extension that builds an HTML gallery of examples from any set of Python scripts.
To contribute a new tutorial, here are the steps to follow:
Create a notebook styled python file. If you want it executed while inserted into documentation, save the file under
examples/tutorials/. If your tutorial contains other auxiliary scripts which are not intended to be included into documentation, save them under
The syntax to write a “notebook styled python file” is very simple. In essence, you only need to write a slightly well formatted python file. Here is a useful guide of how to structure your Python scripts for Sphinx-Gallery.
Put the tutorials into
docs/source/tutorials.rst. You should add it both in
toctree(to make it appear in the sidebar content table), and
cardlinkitem(to create a card link), and specify the appropriate
background(for image) and
linkare the generated link, which is usually
tutorials/<your_python_file_name>.html. Some useful images can be found in
docs/img/thumbnails, but you can always use your own. Available background colors are:
In case you prefer to write your tutorial in jupyter, you can use this script to convert the notebook to python file. After conversion and addition to the project, please make sure the sections headings etc are in logical order.
Build the tutorials. Since some of the tutorials contain complex AutoML examples, it’s very inefficient to build them over and over again. Therefore, we cache the built tutorials in
docs/source/tutorials, so that the unchanged tutorials won’t be rebuilt. To trigger the build, run
make en. This will execute the tutorials and convert the scripts into HTML files. How long it takes depends on your tutorial. As
make enis not very debug-friendly, we suggest making the script runnable by itself before using this building tool.
Some useful HOW-TOs in writing new tutorials:
We only maintain a partial set of documents with translation. Currently, translation is provided in Simplified Chinese only.
If you want to update the translation of an existing document, please update messages in
If you have updated a translated English document, we require that the corresponding translated documents to be updated (at least the update should be triggered). Please follow these steps:
Verify that there are new messages in
Translate the messages.
If you intend to translate a new document:
gettext_documentsinclude your document (probably adding a new regular expression).
See the steps above.
To build the translated documentation (for example Chinese documentation), please run:
If you ever encountered problems for translation builds, try to remove the previous build via
rm -r docs/build/.
Code of Conduct¶
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact firstname.lastname@example.org with any additional questions or comments.
Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.
When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
We enforce every source files in this project to carry a license header. This should be added at the beginning of each file. Please contact the maintainer if you think there should be an exception.
# Copyright (c) Microsoft Corporation. # Licensed under the MIT license.
// Copyright (c) Microsoft Corporation. // Licensed under the MIT license.