Can I use desktop RAM in a PowerEdge T320 server?

No. Non-ECC desktop DDR3 won’t work in a PowerEdge T320, and forcing it can cause boot failures or damage. The T320 needs registered ECC (RDIMM) memory, not unbuffered desktop RAM. Stick with used 8GB or 16GB RDIMMs from eBay.