Pointer Networks

A standard sequence-to-sequence model translating a sentence chooses each output word from a fixed vocabulary of tens of thousands of words — the vocabulary is fixed in advance and doesn't depend on the specific input. That approach breaks down for problems where the correct output is a rearrangement or subset of the input itself, like sorting a list of numbers or finding the shortest tour through a specific set of city coordinates: the "vocabulary" of valid outputs changes with every new input, and can be arbitrarily large.

A pointer network solves this by replacing the usual fixed-vocabulary output layer with an attention mechanism that produces a probability distribution over the input positions rather than over a word list, and then literally "points to" (selects) one input element at each output step. Because the network is choosing among indices into whatever input it's given, it naturally generalizes to inputs of any size without retraining a new output layer, and the mechanism is exactly attention — repurposed as a selection tool instead of a way to summarize context.

A canonical use case is the traveling salesman problem: given a list of city coordinates as input, the pointer network outputs a sequence of pointers back into that same list, defining a visiting order, effectively learning a fast approximate heuristic for a problem that's normally solved by slower combinatorial search algorithms.