In theory, yes. Beware, science follows.
The kinetic energy in the wheel is a measure of how much energy you've spent just getting the wheel to spin rather than investing that energy into the road. Here's the equation for the kinetic energy of a rotating object:
r is radius
m is mass
I is rotational inertia
E is energy
w is angular velocity
P is the constant Pi
E = 1/2 I w2
rotational inertia is obtained by this equation (for hollow cylinders):
I = 1/2 m r2
but w is radians per second, which is rotations per second (rps) times radians per rotation, which is 2 P
w = 2 P rps
rotations per second (rps) is velocity/circumference
circumference is 2 P r
so rps = v/(2 P r) therefore:
w = 2 P (v/(2 P r)) = v/r
This gives us an energy equation of:
E = 1/2 I (v2/r2)
Now we work on the I term, which as stated above is based on mass and radius.
But of course mass depends on radius. The mass of a cylinder is:
m = density 2 P r
Let's let the constant C be "density 2 P":
m = C r
So:
I = C r r2 = C r3
and then
E = 1/2 C r3 (v2/r2)
collapse the 1/2 into the constant, and combine terms, and we get:
E = C r v2
This means that for any given forward speed of the vehicle (v), the energy required just to get the wheel up to speed is greater for larger radius wheels. And this relationship is linear, e.g. a wheel that is 50% larger (in radius) will require 50% more energy to reach the same speed.
So this issue is finally settled, right? Unfortunately not. It takes this much energy to get the wheel up to speed, but once it's there, you only need to add more energy to make up for what has been lost to friction, and here the larger wheel wins.
Larger wheels have less rolling resistance, for several reasons. First, they won't drop (as much) into a smaller hole as a smaller wheel would. Second, they have greater leverage for lifting the wheel over bumps. Third, there is less deformation of the tire at the contact patch on the ground.
The main effect of changing the wheel diameter on a car is the need to change the gears which change the ratio of engine speed to wheel rotation speed; larger wheels clearly rotate more slowly for a given car speed. However, the acceleration and top speed of a car does not depend on the wheel diameter if the gear ratio is optimized for that diameter.
More important is the weight of the wheel. The lighter the wheel, the less is the kinetic energy of the wheel at a given speed and so the more of the work of the engine is used to drive the car forward.
Large diameter wheels and tires effectively put the car in a slightly higher gear, which in principle could be compensated for by having a different set of gears in the power train, but in the real world that change isn't usually cost-free. If your engine is overpowered or "torquey" with its given gear set, a larger tire diameter helps you use that for speed. Conversely many small engines will need small tires to do their (very modest) best acceleration.
The rotary inertia of huge tires can reduce a vehicle's acceleration, but that's only a factor when the wheels and tires are a substantial fraction of the vehicle's mass. Other than monster trucks or beginner-designed RC or robot vehicles, it usually does not matter much.
TL;DR Weight is more of a factor than size due to rotational inertia.
