Anthony Horowitz had said the actor, renowned for playing Nelson Mandela in a recent biopic, was not "suave" enough to be 007.
However, the 60-year-old insisted there was no "colour issue", as he believed Adrian Lester would be a better choice as the first black Bond.
On Twitter, Horowitz said he was "mortified to have caused offence" with his remarks in the Daily Mail at the weekend.
He insisted his choice of the word "street" only related to Elba's role as a detective in the gritty crime drama Luther.
Horowitz admitted it was "a poor choice of word", and went on to stress that he was a writer, not a casting director.
Elba has widely been tipped to succeed Daniel Craig in playing James Bond amid speculation Craig's appearance in upcoming film Spectre, his fourth in the successful franchise, will be his last.
When asked by Esquire magazine if he would be returning to play the troubled secret agent, Craig replied: "I don't know. I really don't know.
"Honestly, I'm not trying to be coy. At the moment I can't even conceive it."
In light of Horowitz's remarks, many fans had expressed their disagreement that Elba was not suave enough to fill Craig's shoes.
One tweeted: "If Idris Elba is 'too street' to be James Bond, then what hope is there for the rest of us? The man could be in pyjamas and outclass us."
The author's latest Bond novel, Trigger Mortis, will be released next week.