You can absolutely do that, but I admit that the UI is not intuitive and not very good. It's one of the oldest features on the site (and most hidden) and is on the chopping block next for a full redesign.
Start in the main library view, then long click / long press on a filament card to enter collection mode. Then you can just click / tap on any other swatches to add them to the collection you're building. Click "Create Collection" when you're done, and you'll get redirected to a link that's just for those swatches. 