The NTRU [Number theory research unit] cryptosystem is a relatively new public key cryptographic algorithm that was first introduced in 1998, and that key runs are much faster than conventional public key algorithms such as RSA, ECC. The main advantage of this cryptosystem is its high speed generation keys, which is often the most important part of public key cryptography. The security of NTRU cryptosystem comes from the interaction of the polynomial mixing system with the independence of reduction modulo two relatively prime integers' p and q. This thesis introduces the concepts behind NTRU as a new public key cryptosystem. NTRU features are reasonably short, easily created keys, high speed, and low memory requirement. These features make it favourable in mobile communication systems, broadcast and satellite channels for its low cost hardware needs. NTRU encryption and decryption use a mixing system suggested by polynomial algebra combined with a clustering principle based on elementary probability theory. Also an approach variant of the NTRU public key cryptosystem called Matrix NTRU cryptosystem is proposed and has been shown to be much faster and have higher efficiency than the classical NTRU cryptosystem. The thesis describes the NTRU Signature Scheme with enhanced document encoding, signature, verification, with provision of documented algorithms and examples. The test and performance analysis performed using a PC with the following specification (processor 1.7 dual cores,memory 512 MB with windows XP-SP2 operating system), and all programs are developed in Visual Basic.