First steps towards supercharging remote development
| Autor: | Córdoba, Carlos, Gerlach, C.A.M. |
|---|---|
| Přispěvatelé: | Montoya, Andres, Clary, Ryan |
| Jazyk: | angličtina |
| Rok vydání: | 2023 |
| Předmět: | |
| DOI: | 10.5281/zenodo.8137350 |
| Popis: | Spyder is a popular scientific environment and IDE among researchers, engineers and data analysts. Over the past few years, a common request from its users has been to support remote and cloud-based workflows, with data and code on high performance computing (HPC) clusters or cloud providers like AWS, GCP and DigitalOcean. Adding such a feature would open up many new research possibilities by enabling the scientific community to tackle data and compute-intensive programming tasks from the ease and efficiency of their local development environment. When users cannot connect to a server through a web browser or want to use an IDE for its reproducibility/reusuability features, they have two options to work remotely. They can either edit and execute their files in a terminal, which is not user-friendly; or start their IDE on the server and display it locally, which is resource-intensive on the remote machine and slow on the local one. Furthermore, scientists typically find it difficult to sync files to the local machine and/or version control their code on the remote one, so remote changes are often lost, out of sync or difficult to integrate with code developed locally. To address this situation, we are adding and enhancing support for local installations of Spyder to sync files and run code on remote compute servers. That will allow users to develop code from their local machine with a desktop IDE and the full computing power of a remote system at their fingertips, while efficiently using server resources. The first step in that direction is to enhance the existing remote code execution in Spyder. The current workflow for that is error-prone and quite cumbersome. It requires users to install the Spyder-Kernels package on the remote machine, manually start a kernel on it and provide the kernel connection file to the local machine. Since Spyder does not currently have a way to store or identify state changes, users will have to repeat this costly setup procedure if there are any configuration modifications on the remote machine. The poster details the plans to fully automate this process so that Spyder can start and connect to a remote kernel automatically, with the user only needing to provide the SSH credentials and some remote environment details. It presents our rationale to make Spyder an IDE with first-class support for remote development, then it will then showcase the new functionality described above, and finally outline our next steps to further improve this workflow for users. Funded by a Chan Zuckerberg Initiative Essential Open Source Software for Science grant - Cycle 5 |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|