Recently I’ve been getting into embedded Linux, particularly the Raspberry Pi and have consequently been learning Python. I really don’t like programming directly on these small devices since the environment is typically spare and slow.
At first I used Cyberduck to open and update files along with a text editor. That became exhausting particularly as my project began to grow and span multiple directories. It was also a lot of Alt+Tabbing back and forth between Cyberduck, the text editor, and PuTTY.
What I really needed was something that I could work on in my main dev environment but deploy and execute on the RPi. This is particularly important to me since my RPi is way out in the garage. It also needed to be able to run as root because all RPi GPIO requires root privileges.
I decided on PyCharm, since I have a JetBrains Toolbox subscription and I’m familiar with CLion which uses the same base IDE framework. Most importantly, PyCharm has a remote debugging feature which coupled with automatic deployment makes everything super easy.
Setting Up Remote Debugging
Below is how I set up my environment. Much of this is found in the PyCharm help documentation. You can find how to set up remote debugging particularly the section on setting up a remote interpreter via SSH.
Automatic Deployment
First we need to setup automatic deployment of our files to the RPi. This part isn’t strictly required but if you don’t do it you’ll have to manage uploading your changes.
- Create a new project called RemoteDebugEx.
- Click on File -> Settings. Then go to Editor -> Code Style and change line endings to “Unix and OS X (\n)”. You want this since your target environment is Linux and different line endings may cause issues.
- Click on Tools -> Deployment -> Configuration.
- Add a new SFTP connection called “My RPi”. This will use SSH to copy files to and from the Pi.
- Provide connection settings. You can use a server name or IP address.
- Press the “Test SFTP connection…” button to make sure everything is correct. You may be prompted to accept the new fingerprint. Click Yes.
- Set paths to say where we want to keep this on our remote. You can choose temporary or permanent locations. (I use temporary, but PyCharm’s sync features allow you to make changes on the RPi and download them back to your computer.)
- Now click Tools -> Deployment -> Automatic Upload. This makes it so files will automatically upload whenever you save.
Remote Interpreter
The next task is to set up a remote interpreter. In this section I’ll also show you how to download changes you make on the Pi.
- Now we need a virtual environment on the remote. Create an empty
requirements.txt
file. Save and it should be uploaded automatically. This also creates the remote debug folder. - Log into the Pi using SSH (I like PuTTY but there are other options). We should be able to see our newly uploaded file.
[shell]ls /home/pi/remote_debug/remote_debug_ex[/shell]
- Create the virtual environment. First make sure virtualenv is installed.
[shell]sudo pip install virtualenv[/shell]
Then navigate to your project root folder on the remove and create the virtual environment.
[shell linenumbers="false"]
cd /home/pi/remote_debug/remote_debug_ex
virtualenv venv
source venv/bin/activate
pip install RPi.GPIO
pip freeze > requirements.txt
[/shell] - Now you want to get the updated
requirements.txt
file back to your computer. Right click the file and choose “Sync with Deployed to My RPi…”. You’ll see a compare dialog that allows you to copy the file back to your computer.
- The next step is to add the virtual environment as a project interpreter. Click on File -> Settings.
- Then go to Project: RemoteDebugEx -> Project Interpreter. Click the gear icon and choose Add Remote.
- Choose to use SSH credentials and enter your host and login information. For “Python interpreter path”, choose the “python” file within your virtual environment folder.
Important: The login you use here is the credentials that the remote process will be run as. You can use the “pi” user, as we have another way of gaining root privileges to access GPIO detailed below.
Run Configuration
Still with me? Now that we’ve got deployment set up and an interpreter that will use our remote virtual environment, the final step is to create a run configuration to actually run a script.
- Create a simple python script and call it
hello_world.py
. Give it the following contents.
[python linenumbers="false"]
import RPi.GPIO
print "hello world!"
[/python] - Click in the upper right of the main window and choose “Edit Configurations…”.
- Click the plus button and choose Python.
- Give the new configuration the name “hello_world (remote)”. For the script, choose the script we just created. For Python interpreter, choose the remote interpreter we created in the last section.
- Now add a path mapping to map from your local project path to the remote path. This lets the interpreter find the source file for what’s executing remotely.
- Save the new configuration and click the run button. You should see PyCharm connect and the hello world print on the debug console.
Running as Root
Rather than enabling logging in as root over SSH, there’s another approach that will work without opening that security hole. Using permissions, we can cause our python interpreter to simply run as root.
- Reconnect using PuTTY and navigate to the project root folder.
[shell]cd /home/pi/remote_debug/remote_debug_ex[/shell]
- Change ownership of the python interpreter to root and cause it to be executed as its owner whenever it’s run.
[shell linenumbers="false"]
sudo chown -v root:root venv/bin/python
sudo chmod -v u+s venv/bin/python
[/shell]
This will cause pip to act a bit funny when you want to install anything later on so just reverse the above changes from step 2 as needed. Just use the same commands but with pi instead of root and with u-s
.
I have a couple of scripts that I keep in the project root for just this purpose. You can download them here.
Conclusion
Now you have a way to remote debug your RPi with the interpreter running as root, allowing access to the Pi’s GPIO. You can do this with multiple projects and even have multiple projects or instances of projects open and debugging remotely.
PyCharm is a great IDE and I encourage you to look into using it to improve your development environment. For example, check out Zeal and the Dash plugin that will cause PyCharm to perform a documentation lookup when you press Ctrl+Shift+D. There are also plugins to provide support for Markdown and bash scripts.
Great tutorial, you just saved me a lot of time, thanks!!
Very helpful. Thank you so much..
This is a great post; but I got lost along the way. I had to back up to where pip failed to install RPi.GPIO:
sudo pip uninstall RPi.GPIO
then install python-dev (I am running debian jessie lite):
sudo apt-get update
sudo apt-get install -t jessie python-dev
then the install for GPIO was successful. However, when I ran it remotely from PyCharm, the console said ‘no module named RPi.GPIO’. I ran SSH into the pi and I ran hello_world.py without a problem. Seems PyCharm is trying to run this script from a different location (venv/bin), and sure enough, the script does not work from there over SSH for me either.
I’m fuzzy on this virtual env thing. Do you need it?
thanks for any help!!!
Mike
@Mike: Thanks for the question. There are a lot of details to the setup so you do have to be careful. When you’re setting up the remote interpreter you have to make sure you set the interpreter path correctly. Look closely again at the last step in the “Remote Interpreter” section. It’s also possible that I missed something in the instructions so let me know if you still have problems.
You don’t technically need to run using virtual environments but many people use them as a matter of course. It’s an alternative to installing all dependencies globally. You have one place where you can see all the dependencies for the project including which version of python. This makes sure things will run correctly when you have several projects that may need different versions of the same package. It also makes it easy to uninstall a project since it and all of its dependencies are co-located.
Mr. Jones:
On this part of your instructions you never mentioned WHERE do I create this file: requirements.txt? in Pycharm or in RPi using putty? also in which folder? Inside or outside RemoteDebugEx?
YOU WROTE:
Remote Interpreter
The next task is to set up a remote interpreter. In this section I’ll also show you how to download changes you make on the Pi.
Now we need a virtual environment on the remote. Create an empty requirements.txt file. Save and it should be uploaded automatically. This also creates the remote debug folder
Put it in the root folder of the project. It’s a python convention. You
pip freeze
your list of packages into requirements.txt and then pip later consumes that file to restore your packages in the virtual environment.Nathan
Quick question: Can you do this just using a Python2.7 on RPI? I have a GUI python script and I am having difficulty running it as root. Is this why you are using a virtual environment? Please, let em know.
The virtual environment is so you can set just that project’s copy of python to automatically run as root. You wouldn’t want your system’s python to always run as root.
There’s not really any magic to this setup. All that’s really going on in the background is PyCharm is logging in via SSH, uploading some debugging helpers, and executing Python in debug mode.
I’m not sure you could remote debug a GUI app though since that would need to be started in a context where the GUI could be shown. Unless your UI is web-based there aren’t any easy options.
What a great tutorial, without any pitfalls! It works like a charm and is fun to follow them. Thanks a lot!
hi Nathan you write an awesome tutorial.
thank you
after all works, when i want to run the programm remotely , i get an bellow error,
“”””””””””””
Error running ‘hello_world(remote)’: Cannot run program “ssh:\pi@192.168.43.25:22\home\pi\remote_debug\remote_debug_ex\venv” (in directory “C:\Users\reza\PycharmProjects\RemoteDebugEx”): CreateProcess error=2, The system cannot find the file specified
“”””””””””””
what is it about ?
in section “Remote Interpreter” and step 7
when i set the “Python interpreter path” to “/home/pi/remote_debug/remote_debug_ex/venv”
it shows me an error and said : “”Can’t create python SDK, Interpreter ‘/home/pi/remote_debug/remote_debug_ex/venv’ doesn’t exist on remote server”
Hi, thanks for the great tutorial!
I have a question about module usage though, how do I import modules so that they are available in pycharm as well? I tried to install an adafruit DHT module, and I couldn’t get it to work but the next day somehow pycharm could find it.
But now that I need another module, I just can’t get it to work.
It seems that smbus is available on the pi, but not when I’m in pycharm :
https://i.imgur.com/0ohmz0F.png
I tried to make it available via :
cd /home/pi/remote_debug/remote_debug_ex
source venv/bin/activate
sudo apt-get install python-smbus
pip freeze > requirements.txt
But as you can see in the image, it was already installed.
How would you sync dependencies and make them available both to pycharm and the pi?
Thanks again