Omit backup of a file group - is it possible?

I have a table in my database that holds binary data. I have created this table on a separate file group so that my primary file does not fill up. This table on the secondary file group have a foreign key into a tabel from the primary file group.
When I do backups, is it possible to omit the secondary file group (*.ndf file) to be included in the back up? So essentially I only want the PRIMARY filegroup (*.mdf & *.ldf files) to be backed up.
...