Did you know about the Color Picker hotkey in the Sims 4? It's a must hotkey when using the swatch color. Here's a youtube short about it.
It's really simple. Before you place an object down, you can click the "+" on the keyboard. This will open up the color swatch. I also mentioned another shortcut. If an object is already placed, you can press the R button and your cursor will change into the swatch cursor, from there, you can click on an item and the swatch color palette will open.
I hope you find this helpful.
Dag. Dag.