Let $r_1$ and $r_2$ be the zeros of the polynomial $x^2 - x - 1$
Therefore,
$r_1 + r_2 = 1$ $...(i)$
$r_1 r_2 = -1$ $...(ii)$
Since $(x - r_1)$ and $(x - r_2)$ divide $x^2 - x - 1$, they also divide $ax^{17} + bx^{16} + 1$
Therefore,
$a{r_1}^{17} + b{r_1}^{16} + 1 = 0$
$\Rightarrow b = \dfrac{ - 1 - a{r_1}^{17}}{{r_1}^{16} }$
-----------book page break-----------
and,
$a{r_2}^{17} + b{r_2}^{16} + 1 = 0$
$\Rightarrow b = \dfrac{- 1 -a{r_2}^{17}}{{r_2}^{16} }$
Therefore,
$\dfrac{ - 1 - a{r_1}^{17}}{{r_1}^{16} } = \dfrac{- 1 -a{r_2}^{17}}{{r_2}^{16} }$
$\Rightarrow - {r_2}^{16} - a{r_1}^{17}{r_2}^{16} = - {r_1}^{16} -a{r_2}^{17}{r_1}^{16}$
$\Rightarrow a \left( {r_2}^{17}{r_1}^{16} - a{r_1}^{17}{r_2}^{16} \right) = {r_2}^{16} - {r_1}^{16}$
$\Rightarrow a = \dfrac{{r_2}^{16} - {r_1}^{16}}{{r_2}^{17}{r_1}^{16} - a{r_1}^{17}{r_2}^{16}}$
$\Rightarrow a = \dfrac{{r_2}^{16} - {r_1}^{16}}{ {r_1}^{16}{r_2}^{16} \left( {r_2} - {r_1} \right)}$
$\Rightarrow a = \dfrac{\left({r_2} - {r_1}\right) \left({r_2} + {r_1}\right) \left({r_2}^{2} + {r_1}^{2}\right) \left({r_2}^{4} + {r_1}^{4}\right) \left({r_2}^{8} + {r_1}^{8}\right)}{ {r_1}^{16}{r_2}^{16} \left( {r_2} - {r_1} \right)}$
The discriminant of the $x^2 - x - 1 = 0$ is $\sqrt{5}$ therefore $r_1 \ne r_2$
Therefore,
$a = \dfrac{ \cancel{\left({r_2} - {r_1}\right)} \left({r_2} + {r_1}\right) \left({r_2}^{2} + {r_1}^{2}\right) \left({r_2}^{4} + {r_1}^{4}\right) \left({r_2}^{8} + {r_1}^{8}\right)}{ {r_1}^{16}{r_2}^{16} \cancel{\left( {r_2} - {r_1} \right)}}$
-----------book page break-----------
$a = \dfrac{ \left({r_2} + {r_1}\right) \left({r_2}^{2} + {r_1}^{2}\right) \left({r_2}^{4} + {r_1}^{4}\right) \left({r_2}^{8} + {r_1}^{8}\right)}{ ({r_1}{r_2})^{16} }$
Using equations $(i)$ and $(ii)$,
${r_1}^2 + {r_2}^2 = ({r_1} + {r_2})^2 - 2{r_1}{r_2} = 1 + 2 = 3$
${r_1}^4 + {r_2}^4 = ({r_1}^2 + {r_2}^2)^2 - 2{r_1}^2{r_2}^2 = (3)^2 - 2(-1)^2 = 7$
${r_1}^8 + {r_2}^8 = ({r_1}^4 + {r_2}^4)^2 - 2{r_1}^4{r_2}^4 = (7)^2 - 2(-1)^2 = 47$
Therefore,
$a = \dfrac{ 1 \cdot 3 \cdot 7 \cdot 47}{ (-1)^{16} } = 987$