If you are using unroasted vermicelli, heat 1 tbsp of ghee and roast the vermicelli, over medium heat, till it firsts turns white and then golden brown. Do not let it brown too much. Remove from the pan and keep aside.
Heat the remaining ghee in the same pan and, over low heat, fry the raisins till they plump up. Remove and keep aside. In the same ghee, fry the broken cashewnuts till they turn uniformly golden brown. Do not let them become dark brown. Remove them from the pan and keep aside.
In another heavy bottomed pan, pour the milk and bring it to boil. Add the roasted vermicelli and stir well. Allow the vermicelli-milk mixture to come to a boil, stirring frequently and then let it simmer until the vermicelli has cooked well. Stir frequently so that the vermicelli does not settle to the bottom of the pan, clump and burn.
Once the vermicelli has cooked and the milk has thickened a bit (this should take about 20 minutes or so), add the sugar and stir. Let it cook for a further 5 to 10 minutes. The consistency of the payasam should be a bit thick but still reasonably liquid when you stir it. It will thicken a little more once it cools. Add the cardamom, cashewnuts and raisins, stir a couple of times and take it off the heat.
Let it cool, stirring occasionally to prevent a skin from forming on the top. Serve warm or let it chill before serving if you prefer.