Some old dry sump engines (air cooled Porsche 911 engines) required checking the oil level while the engine is running. So you would need to remove the oil cap to the oil reservoir to add oil if needed, which would cause a vacuum leak with the associated rough running and low idle (think pre drive by wire). You'd have to manually idle up the engine using the idle screw on the throttle body to within spec, prior to taking an oil level reading. Then once done, close the oil cap and readjust the idle.
Pretty much all other engines I've ever come across require the engine to be turned off to check the oil level. So there really is no reason, or at least one that I can think of, to remove the oil cap while the engine is running. If you notice that there is no change in behavior when you remove the oil cap while the engine is running, then that would be a cause for concern.