I don't know which would give me more performance boost: OC-ing the GPU or VRAM ?
Both and neither. It depends on the game, the monitor (resolutions) and the settings you use.
The actual GPU is a 'Cape Verde' chip. It is the hearth which does your calculations. If it is not fast enough for your game but the memory is more than fast enough then overclocking it will help. Probably even lineair. (That is a 10% OC might yield 10% performance gain).
On the other edge the CPU might be fast enough but spent most of its time waiting for memory. (E.g. when playing UHD or 4K with 8x antialiassing). In that case ramping up CPU speed will not help much, but increasing bandwidth to the memory will.
And lastly, there are many situations which fall in between these two, and they are not mutually exclusive. As a result it is usually best to OC both.
There are ofc exceptions. Some cards are sold with a weak GPU chip and lots of slow memory (which usually does not make any sense on an average performance scale, but people tend to buy graphics cards with as much memory as possible in the vague but false idea that this 'also must be the fast cards'.
The HD7750 is not one of those. It had a fast memory hungry GPU and proper GDDR5 (fast!) memory. Thrust the GPU team (aka AMD) to have done its research on this and boost both to keep that balance.
Side note 1: If you play real time games then turning down the resolution might help a lot to smoothen the screen. E.g. say your native resolution is 2560x1440. Set the game to only 1280x720. That way the card has to do only ¼th of the calculations.
Side note 2: Keep an eye on power consumption. Make sure that your PSU can handle the extra +12v needed before you OC. Ditto cooling.
