Rakshasa (राक्षस) & Yaksha (यक्ष)

As mentioned in Ramayana – Uttarakand – Chaturath Sarga ( चतुर्थ​ सर्ग )

When Lord Ram asked the Rishi Augusta,  that who were the Rakshasa born before the Pulastya (पुलस्त्य) lineage  ( referred to Ravana, Kumbhakaran e.t.c )

Rishi Augusta replied him , that in the beginning when Lord Brahma created these Ocean on the earth and is further created the species living in them.

प्रजापतिः पुरा सृष्ट्वा अपः सलिलसम्भवः |
तासां गोपायनो सत्त्वानसृजत्पद्मसम्भवः || 9||

ते सत्त्वाः सत्त्वकर्तारं विनीतवदुपस्थिताः |
किं कुर्म इति भाषन्तः क्षुत्पिपासा भयार्दिताः || 10||

Soon all those species who were surviving in those waters, being affected by hunger and thirst went to Lord Brahma to seek guidance.

प्रजापतिस्तु तान्याह सत्त्वाहि प्रहसन्निव |
आभाष्य वाचा यत्नेन रक्षध्वमिति मानदः || 11||

Lord Brahma directed them to protect the waters created by him, and survive on them

रक्षाम इति तत्रान्यैर्यक्षामेति तथापरैः |
भुङ्क्षिताभुङ्क्षितैरुक्तस्ततस्तानाह भूतकृत् || 12||

Among them some species said in reply that “they will protect the waters” and some said that “they will worship waters considering it as sacred”

रक्षाम इति यैरुक्तं राक्षसास्ते भवन्तु वः |
यक्षाम इति यैरुक्तं ते वै यक्षा भवन्तु वः || १३||

Lord Brahma said – among you “those who have taken oath as to protect are known as Rakshasa” (रक्षाम इति ये उक्तं ते राक्षसा:) and “those who will worship will be know as Yaksha”(यक्षाम इति ये उक्तं ते यक्षा:)

Who were the first Rakshasa’s (राक्षस)?

Among species created by Lord Bharama, first were brother known as Haiti (हेतिः) and Prahaiti (प्रहेति:)

Haiti (हेतिः) married to sister of Kala (काल​) named as Bhaya (भया), they both had son called Vidyutkesh (विद्युत्केश)

Vidyutkesh (विद्युत्केश) married to daughter of Sandhya (सन्ध्या) named as Salakatkanta (सालकटङ्कटा), and they had a son called Sukesh (सुकेश)

Sukesh (सुकेश) married to daughter of Garamini Gandharva (ग्रामणी) , her name was Devwati (देववती), they has three sons Malayvaan (माल्यवान्) , Sumali (सुमाली) and Mali (माली)

These three brother were first to made Lanka as capital, after being defeated by Lord Vishnu in a battle, the Malayvaan and Sumali went to Patalalok. Mali was killed in the battlefield. 

Kekasi (कैकसी ) was the daughter of Sumali (सुमाली), who married to Vishrava and from her the Ravana,  Kumbhakaran , Vibhishan were born.