Prime Minister Abdalla Hamdok and rebel chief Abdelaziz al-Hilu signed the peace deal in neighbouring Ethiopia, Sudan's official news agency SUNA said, which posted pictures of the two men smiling and shaking hands.
The deal is significant because Hilu, a veteran guerilla fighter who leads a faction of the Sudan People's Liberation Movement-North (SPLM-N), had been one of two rebel leaders who had rejected an earlier peace deal with Khartoum.
While Khartoum and Hilu agreed to a "cessation of hostilities", the deal allows rebels to keep hold of their guns for "self-protection" until Sudan's constitution is changed to separate religion and government.
"For Sudan to become a democratic country where the rights of all citizens are enshrined, the constitution should be based on the principle of 'separation of religion and state', in the absence of which the right to self-determination must be respected," said a copy of the deal, confirmed to AFP by rebel spokesman Jack Mahmoud.
The agreement, signed late Thursday in Ethiopia's capital Addis Ababa, follows a deal signed on Monday in South Sudan's capital Juba with leaders of a coalition of rebel forces, the Sudan Revolutionary Front (SRF).
The SRF brings together rebels from the war-ravaged western Darfur region, as well as the southern states of Blue Nile and South Kordofan, where Hilu's SPLM-N is based.
Hilu's stronghold in the Nuba Mountains of South Kordofan has a significant Christian community among its mainly non-Arab population and he has long championed a secular state to replace the Islamist regime of military strongman Omar al-Bashir which was overthrown in April last year.
The latest agreement means only one key group remains still fighting, a wing of the Darfur-based Sudan Liberation Movement (SLM) led by Paris-based Abdelwahid Nour.
Fighting in Darfur alone left around 300,000 people dead after rebels took up arms in 2003.
Conflict in South Kordofan and Blue Nile erupted in 2011 as South Sudan seceded, resuming a war that had raged from 1983-2005.
Rebels fought troops deployed by Bashir, who is wanted by the International Criminal Court (ICC) over charges of genocide and crimes against humanity in the Darfur conflict.
While past peace deals have rapidly crumbled, ending war with the rebels has been a cornerstone policy of Sudan's transitional government, which came to power in the months after Bashir's overthrow.