Can I mix the yellow ochre and the red together when doing the limited palette exercise?

Yes. You can mix the yellow ochre and the red together. You can also mix the white, black, red and the yellow ochre together. Just with these four colors, you can make an infinite number of colors and values by mixing some or all of them together. The colors in your art will usually look better if many of the colors have some of the other colors mixed with them.