By the sounds of it the Wifi is the issue. Bandwidth tends to fluctuate & be unreliable over wireless transmissions (how badly depends on variables of set up & equipment, etc.) Then you also have possible interference between from radio magnetic devices such as lights, electronic devices, etc.

But I am confused. You say Wifi, yet you say you are connected via Cable. So it's a wireless modem/router connected to the cable right?

'If' it is possible to hard wire it ... meaning using ethernet cable instead of wireless, try that & I'm sure you will find your speeds will be fast & reliable. If you find that's the case, you may need to look at possibility of either keeping it connected via ethernet else, looking at a secondary wifi send point closer to where you will use your computer.

Other than that, I am aware there are 'some' Internet providers in Japan that shape p2p sharing bandwidth.