fbt/sbase
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
ba2f04f391
sbase/sort.c

438 lines
9.2 KiB
C
Raw Normal View History

add sort
2011-06-02 12:03:34 +00:00
/* See LICENSE file for copyright and license details. */
sort: add -k, fix -u Options that are specific to a single key definition are not supported (e.g. "sort -k 2,3n -k 4,4"). Should you try to specify such definitions, sort will return with EXIT_FAILURE and an error message. Instead, all key definitions exclusively use the global settings. It always behaves like -b was set.
2014-04-12 15:53:10 +00:00
#include <ctype.h>
add sort
2011-06-02 12:03:34 +00:00
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
Fix coding style It was about damn time. Consistency is very important in such a big codebase.
2014-11-13 17:29:30 +00:00
Use queue.h-macros in sort(1) This is much easier to read than having yet another handrolled list implementation. Tested and more or less clearly equivalent. Now that I have uni-vac, I'll have enough time to refactor more.
2015-08-02 21:55:54 +00:00
#include "queue.h"
add sort
2011-06-02 12:03:34 +00:00
#include "text.h"
Make sort(1) utf-compliant and update README Make it clear that <blank> characters just are spaces or tabs and not a special group which needs special treatment for wide characters. Also, and that was the only problem here, correctly calculate the offset given by the key definitions for the start- and end-characters using libutf-utility-functions. Mark the progress in the README and put parentheses around the missing flags which are insane to implement for no real gain.
2015-08-03 15:35:01 +00:00
#include "utf.h"
add sort
2011-06-02 12:03:34 +00:00
#include "util.h"
sort: add -k, fix -u Options that are specific to a single key definition are not supported (e.g. "sort -k 2,3n -k 4,4"). Should you try to specify such definitions, sort will return with EXIT_FAILURE and an error message. Instead, all key definitions exclusively use the global settings. It always behaves like -b was set.
2014-04-12 15:53:10 +00:00
struct keydef {
sort: work with signed integers as well
2014-05-06 11:35:06 +00:00
int start_column;
int end_column;
int start_char;
int end_char;
sort: add support for "per-keydef" flags
2014-05-06 14:07:05 +00:00
int flags;
Use queue.h-macros in sort(1) This is much easier to read than having yet another handrolled list implementation. Tested and more or less clearly equivalent. Now that I have uni-vac, I'll have enough time to refactor more.
2015-08-02 21:55:54 +00:00
TAILQ_ENTRY(keydef) entry;
sort: add support for "per-keydef" flags
2014-05-06 14:07:05 +00:00
};
sort: Fix line comparison when col buffer contains data from longer line I'm not sure if there are other implications of this or not, but the issue is that columns() uses len to store the allocated buffer size, but linecmp() compares up to len bytes. If those trailing bytes do not match, the line is considered not matching, even though the relevant parts of the buffer do match. To resolve this, also keep track of column capacity. Additionally, since there is no reason to keep the existing data when resizing, just use free and emalloc rather than erealloc. The simplest case I could reduce it to is this: if [ "$(printf '%s\n' a a xxb xxc | ./sort -u)" = "$(printf '%s\n' a xxb xxc)" ] ; then echo pass else echo fail fi
2016-05-15 01:56:54 +00:00
struct column {
struct line line;
size_t cap;
};
sort: add support for "per-keydef" flags
2014-05-06 14:07:05 +00:00
enum {
Use queue.h-macros in sort(1) This is much easier to read than having yet another handrolled list implementation. Tested and more or less clearly equivalent. Now that I have uni-vac, I'll have enough time to refactor more.
2015-08-02 21:55:54 +00:00
MOD_N = 1 << 0,
MOD_STARTB = 1 << 1,
MOD_ENDB = 1 << 2,
MOD_R = 1 << 3,
Add -d, -f and -i flags to sort(1) Here's the patch with updated manpage and usage().
2016-02-16 00:42:25 +00:00
MOD_D = 1 << 4,
MOD_F = 1 << 5,
MOD_I = 1 << 6,
sort: add -k, fix -u Options that are specific to a single key definition are not supported (e.g. "sort -k 2,3n -k 4,4"). Should you try to specify such definitions, sort will return with EXIT_FAILURE and an error message. Instead, all key definitions exclusively use the global settings. It always behaves like -b was set.
2014-04-12 15:53:10 +00:00
};
Use queue.h-macros in sort(1) This is much easier to read than having yet another handrolled list implementation. Tested and more or less clearly equivalent. Now that I have uni-vac, I'll have enough time to refactor more.
2015-08-02 21:55:54 +00:00
static TAILQ_HEAD(kdhead, keydef) kdhead = TAILQ_HEAD_INITIALIZER(kdhead);
sort: add -k, fix -u Options that are specific to a single key definition are not supported (e.g. "sort -k 2,3n -k 4,4"). Should you try to specify such definitions, sort will return with EXIT_FAILURE and an error message. Instead, all key definitions exclusively use the global settings. It always behaves like -b was set.
2014-04-12 15:53:10 +00:00
sort: Implement -c and -C flags
2014-11-23 19:35:56 +00:00
static int Cflag = 0, cflag = 0, uflag = 0;
add -t flag to sort
2014-05-15 18:08:17 +00:00
static char *fieldsep = NULL;
Audit sort(1) and mark it as finished 1) Remove the function prototypes. No need for them, as the functions are ordered. 2) Add fieldseplen, so the length of the field-separator is not calculated nearly each time skipcolumn() is called. 3) rename next_col to skip_to_next_col so the purpose is clear, also reorder the conditional accordingly. 4) Put parentheses around certain ternary expressions. 5) BUGFIX: Don't just exit() in check(), but make it return something, so we can cleanly fshut() everything. 6) OFF-POSIX: Posix for no apparent reason does not allow more than one file when the -c or -C flags are given. This can be problematic when you want to check multiple files. With the change 5), rewriting check() to return a value, I went off-posix after discussing this with Dimitris to just allow arbitrary numbers of files. Obviously, this does not break scripts and is convenient for everybody who wants to quickly check a big amount of files. As soon as 1 file is "unsorted", the return value is 1, as expected. For convenience reasons, check()'s warning now includes the filename. 7) BUGFIX: Set ret to 2 instead of 1 when the fshut(fp, *argv) fails. 8) BUGFIX: Don't forget to fshut stderr at the end. This would improperly return 1 in the following case: $ sort -c unsorted_file 2> /dev/full 9) Other style changes, line length, empty line before return.
2015-08-04 10:45:59 +00:00
static size_t fieldseplen = 0;
sort: Fix line comparison when col buffer contains data from longer line I'm not sure if there are other implications of this or not, but the issue is that columns() uses len to store the allocated buffer size, but linecmp() compares up to len bytes. If those trailing bytes do not match, the line is considered not matching, even though the relevant parts of the buffer do match. To resolve this, also keep track of column capacity. Additionally, since there is no reason to keep the existing data when resizing, just use free and emalloc rather than erealloc. The simplest case I could reduce it to is this: if [ "$(printf '%s\n' a a xxb xxc | ./sort -u)" = "$(printf '%s\n' a xxb xxc)" ] ; then echo pass else echo fail fi
2016-05-15 01:56:54 +00:00
static struct column col1, col2;
sort: Move lines and nlines globals into a struct
2012-05-21 20:09:44 +00:00
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
static void
Rename struct linebufline to struct line and add linecmp() This simplifies the handling in sort(1) and comm(1) quite a bit.
2016-03-07 01:04:29 +00:00
skipblank(struct line *a)
Factor out parse_keydef() into addkeydef() and reorder functions Add a small comment explaining the data-structure and sort the functions according to usage, not alphabetically.
2015-08-02 23:12:29 +00:00
{
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
while (a->len && (*(a->data) == ' ' || *(a->data) == '\t')) {
a->data++;
a->len--;
}
Factor out parse_keydef() into addkeydef() and reorder functions Add a small comment explaining the data-structure and sort the functions according to usage, not alphabetically.
2015-08-02 23:12:29 +00:00
}
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
static void
Rename struct linebufline to struct line and add linecmp() This simplifies the handling in sort(1) and comm(1) quite a bit.
2016-03-07 01:04:29 +00:00
skipnonblank(struct line *a)
sort: add -k, fix -u Options that are specific to a single key definition are not supported (e.g. "sort -k 2,3n -k 4,4"). Should you try to specify such definitions, sort will return with EXIT_FAILURE and an error message. Instead, all key definitions exclusively use the global settings. It always behaves like -b was set.
2014-04-12 15:53:10 +00:00
{
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
while (a->len && (*(a->data) != '\n' && *(a->data) != ' ' &&
*(a->data) != '\t')) {
a->data++;
a->len--;
}
Reorder functions in sort(1) I kind of missed that the sorting was still not properly done. parse_flags() and addkeydef() are independent of everything else, so they can be put at the bottom. Sorting the other functions reveals the true hierarchy much better.
2015-08-02 23:32:21 +00:00
}
sort: Implement -c and -C flags
2014-11-23 19:35:56 +00:00
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
static void
Rename struct linebufline to struct line and add linecmp() This simplifies the handling in sort(1) and comm(1) quite a bit.
2016-03-07 01:04:29 +00:00
skipcolumn(struct line *a, int skip_to_next_col)
Reorder functions in sort(1) I kind of missed that the sorting was still not properly done. parse_flags() and addkeydef() are independent of everything else, so they can be put at the bottom. Sorting the other functions reveals the true hierarchy much better.
2015-08-02 23:32:21 +00:00
{
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
char *s;
Reorder functions in sort(1) I kind of missed that the sorting was still not properly done. parse_flags() and addkeydef() are independent of everything else, so they can be put at the bottom. Sorting the other functions reveals the true hierarchy much better.
2015-08-02 23:32:21 +00:00
if (fieldsep) {
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
if ((s = memmem(a->data, a->len, fieldsep, fieldseplen))) {
sort: Consider end field in keydef when additional fields are present Currently, if the delimiter is found after the last field of a keydef, only up to the beginning of the field is considered. This breaks `sort -k N,N`, as well as whenever the sorted order comes down to that last field. Thanks to Richard Ipsum for the bug report and proposed patch.
2020-01-01 20:16:16 +00:00
if (skip_to_next_col)
Actually move past the field separator Previously, sort(1) failed on key-based sorting and was caught in an infinite loop with the c-flag.
2016-03-06 23:20:24 +00:00
s += fieldseplen;
sort: Consider end field in keydef when additional fields are present Currently, if the delimiter is found after the last field of a keydef, only up to the beginning of the field is considered. This breaks `sort -k N,N`, as well as whenever the sorted order comes down to that last field. Thanks to Richard Ipsum for the bug report and proposed patch.
2020-01-01 20:16:16 +00:00
a->len -= s - a->data;
a->data = s;
Audit sort(1) and mark it as finished 1) Remove the function prototypes. No need for them, as the functions are ordered. 2) Add fieldseplen, so the length of the field-separator is not calculated nearly each time skipcolumn() is called. 3) rename next_col to skip_to_next_col so the purpose is clear, also reorder the conditional accordingly. 4) Put parentheses around certain ternary expressions. 5) BUGFIX: Don't just exit() in check(), but make it return something, so we can cleanly fshut() everything. 6) OFF-POSIX: Posix for no apparent reason does not allow more than one file when the -c or -C flags are given. This can be problematic when you want to check multiple files. With the change 5), rewriting check() to return a value, I went off-posix after discussing this with Dimitris to just allow arbitrary numbers of files. Obviously, this does not break scripts and is convenient for everybody who wants to quickly check a big amount of files. As soon as 1 file is "unsorted", the return value is 1, as expected. For convenience reasons, check()'s warning now includes the filename. 7) BUGFIX: Set ret to 2 instead of 1 when the fshut(fp, *argv) fails. 8) BUGFIX: Don't forget to fshut stderr at the end. This would improperly return 1 in the following case: $ sort -c unsorted_file 2> /dev/full 9) Other style changes, line length, empty line before return.
2015-08-04 10:45:59 +00:00
} else {
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
a->data += a->len - 1;
a->len = 1;
Audit sort(1) and mark it as finished 1) Remove the function prototypes. No need for them, as the functions are ordered. 2) Add fieldseplen, so the length of the field-separator is not calculated nearly each time skipcolumn() is called. 3) rename next_col to skip_to_next_col so the purpose is clear, also reorder the conditional accordingly. 4) Put parentheses around certain ternary expressions. 5) BUGFIX: Don't just exit() in check(), but make it return something, so we can cleanly fshut() everything. 6) OFF-POSIX: Posix for no apparent reason does not allow more than one file when the -c or -C flags are given. This can be problematic when you want to check multiple files. With the change 5), rewriting check() to return a value, I went off-posix after discussing this with Dimitris to just allow arbitrary numbers of files. Obviously, this does not break scripts and is convenient for everybody who wants to quickly check a big amount of files. As soon as 1 file is "unsorted", the return value is 1, as expected. For convenience reasons, check()'s warning now includes the filename. 7) BUGFIX: Set ret to 2 instead of 1 when the fshut(fp, *argv) fails. 8) BUGFIX: Don't forget to fshut stderr at the end. This would improperly return 1 in the following case: $ sort -c unsorted_file 2> /dev/full 9) Other style changes, line length, empty line before return.
2015-08-04 10:45:59 +00:00
}
Reorder functions in sort(1) I kind of missed that the sorting was still not properly done. parse_flags() and addkeydef() are independent of everything else, so they can be put at the bottom. Sorting the other functions reveals the true hierarchy much better.
2015-08-02 23:32:21 +00:00
} else {
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
skipblank(a);
skipnonblank(a);
Factor out parse_keydef() into addkeydef() and reorder functions Add a small comment explaining the data-structure and sort the functions according to usage, not alphabetically.
2015-08-02 23:12:29 +00:00
}
Reorder functions in sort(1) I kind of missed that the sorting was still not properly done. parse_flags() and addkeydef() are independent of everything else, so they can be put at the bottom. Sorting the other functions reveals the true hierarchy much better.
2015-08-02 23:32:21 +00:00
}
Factor out parse_keydef() into addkeydef() and reorder functions Add a small comment explaining the data-structure and sort the functions according to usage, not alphabetically.
2015-08-02 23:12:29 +00:00
sort: Fix line comparison when col buffer contains data from longer line I'm not sure if there are other implications of this or not, but the issue is that columns() uses len to store the allocated buffer size, but linecmp() compares up to len bytes. If those trailing bytes do not match, the line is considered not matching, even though the relevant parts of the buffer do match. To resolve this, also keep track of column capacity. Additionally, since there is no reason to keep the existing data when resizing, just use free and emalloc rather than erealloc. The simplest case I could reduce it to is this: if [ "$(printf '%s\n' a a xxb xxc | ./sort -u)" = "$(printf '%s\n' a xxb xxc)" ] ; then echo pass else echo fail fi
2016-05-15 01:56:54 +00:00
static void
columns(struct line *line, const struct keydef *kd, struct column *col)
Reorder functions in sort(1) I kind of missed that the sorting was still not properly done. parse_flags() and addkeydef() are independent of everything else, so they can be put at the bottom. Sorting the other functions reveals the true hierarchy much better.
2015-08-02 23:32:21 +00:00
{
Make sort(1) utf-compliant and update README Make it clear that <blank> characters just are spaces or tabs and not a special group which needs special treatment for wide characters. Also, and that was the only problem here, correctly calculate the offset given by the key definitions for the start- and end-characters using libutf-utility-functions. Mark the progress in the README and put parentheses around the missing flags which are insane to implement for no real gain.
2015-08-03 15:35:01 +00:00
Rune r;
Rename struct linebufline to struct line and add linecmp() This simplifies the handling in sort(1) and comm(1) quite a bit.
2016-03-07 01:04:29 +00:00
struct line start, end;
sort: Fix line comparison when col buffer contains data from longer line I'm not sure if there are other implications of this or not, but the issue is that columns() uses len to store the allocated buffer size, but linecmp() compares up to len bytes. If those trailing bytes do not match, the line is considered not matching, even though the relevant parts of the buffer do match. To resolve this, also keep track of column capacity. Additionally, since there is no reason to keep the existing data when resizing, just use free and emalloc rather than erealloc. The simplest case I could reduce it to is this: if [ "$(printf '%s\n' a a xxb xxc | ./sort -u)" = "$(printf '%s\n' a xxb xxc)" ] ; then echo pass else echo fail fi
2016-05-15 01:56:54 +00:00
size_t utflen, rlen;
Reorder functions in sort(1) I kind of missed that the sorting was still not properly done. parse_flags() and addkeydef() are independent of everything else, so they can be put at the bottom. Sorting the other functions reveals the true hierarchy much better.
2015-08-02 23:32:21 +00:00
int i;
Factor out parse_keydef() into addkeydef() and reorder functions Add a small comment explaining the data-structure and sort the functions according to usage, not alphabetically.
2015-08-02 23:12:29 +00:00
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
start.data = line->data;
start.len = line->len;
for (i = 1; i < kd->start_column; i++)
skipcolumn(&start, 1);
Reorder functions in sort(1) I kind of missed that the sorting was still not properly done. parse_flags() and addkeydef() are independent of everything else, so they can be put at the bottom. Sorting the other functions reveals the true hierarchy much better.
2015-08-02 23:32:21 +00:00
if (kd->flags & MOD_STARTB)
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
skipblank(&start);
for (utflen = 0; start.len > 1 && utflen < kd->start_char - 1;) {
rlen = chartorune(&r, start.data);
start.data += rlen;
start.len -= rlen;
Make sort(1) utf-compliant and update README Make it clear that <blank> characters just are spaces or tabs and not a special group which needs special treatment for wide characters. Also, and that was the only problem here, correctly calculate the offset given by the key definitions for the start- and end-characters using libutf-utility-functions. Mark the progress in the README and put parentheses around the missing flags which are insane to implement for no real gain.
2015-08-03 15:35:01 +00:00
utflen++;
}
Factor out parse_keydef() into addkeydef() and reorder functions Add a small comment explaining the data-structure and sort the functions according to usage, not alphabetically.
2015-08-02 23:12:29 +00:00
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
end.data = line->data;
end.len = line->len;
Reorder functions in sort(1) I kind of missed that the sorting was still not properly done. parse_flags() and addkeydef() are independent of everything else, so they can be put at the bottom. Sorting the other functions reveals the true hierarchy much better.
2015-08-02 23:32:21 +00:00
if (kd->end_column) {
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
for (i = 1; i < kd->end_column; i++)
skipcolumn(&end, 1);
Reorder functions in sort(1) I kind of missed that the sorting was still not properly done. parse_flags() and addkeydef() are independent of everything else, so they can be put at the bottom. Sorting the other functions reveals the true hierarchy much better.
2015-08-02 23:32:21 +00:00
if (kd->flags & MOD_ENDB)
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
skipblank(&end);
Make sort(1) utf-compliant and update README Make it clear that <blank> characters just are spaces or tabs and not a special group which needs special treatment for wide characters. Also, and that was the only problem here, correctly calculate the offset given by the key definitions for the start- and end-characters using libutf-utility-functions. Mark the progress in the README and put parentheses around the missing flags which are insane to implement for no real gain.
2015-08-03 15:35:01 +00:00
if (kd->end_char) {
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
for (utflen = 0; end.len > 1 && utflen < kd->end_char;) {
rlen = chartorune(&r, end.data);
end.data += rlen;
end.len -= rlen;
Make sort(1) utf-compliant and update README Make it clear that <blank> characters just are spaces or tabs and not a special group which needs special treatment for wide characters. Also, and that was the only problem here, correctly calculate the offset given by the key definitions for the start- and end-characters using libutf-utility-functions. Mark the progress in the README and put parentheses around the missing flags which are insane to implement for no real gain.
2015-08-03 15:35:01 +00:00
utflen++;
}
} else {
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
skipcolumn(&end, 0);
Make sort(1) utf-compliant and update README Make it clear that <blank> characters just are spaces or tabs and not a special group which needs special treatment for wide characters. Also, and that was the only problem here, correctly calculate the offset given by the key definitions for the start- and end-characters using libutf-utility-functions. Mark the progress in the README and put parentheses around the missing flags which are insane to implement for no real gain.
2015-08-03 15:35:01 +00:00
}
Reorder functions in sort(1) I kind of missed that the sorting was still not properly done. parse_flags() and addkeydef() are independent of everything else, so they can be put at the bottom. Sorting the other functions reveals the true hierarchy much better.
2015-08-02 23:32:21 +00:00
} else {
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
end.data += end.len - 1;
end.len = 1;
Reorder functions in sort(1) I kind of missed that the sorting was still not properly done. parse_flags() and addkeydef() are independent of everything else, so they can be put at the bottom. Sorting the other functions reveals the true hierarchy much better.
2015-08-02 23:32:21 +00:00
}
sort: Fix line comparison when col buffer contains data from longer line I'm not sure if there are other implications of this or not, but the issue is that columns() uses len to store the allocated buffer size, but linecmp() compares up to len bytes. If those trailing bytes do not match, the line is considered not matching, even though the relevant parts of the buffer do match. To resolve this, also keep track of column capacity. Additionally, since there is no reason to keep the existing data when resizing, just use free and emalloc rather than erealloc. The simplest case I could reduce it to is this: if [ "$(printf '%s\n' a a xxb xxc | ./sort -u)" = "$(printf '%s\n' a xxb xxc)" ] ; then echo pass else echo fail fi
2016-05-15 01:56:54 +00:00
col->line.len = MAX(0, end.data - start.data);
if (!(col->line.data) || col->cap < col->line.len + 1) {
free(col->line.data);
col->line.data = emalloc(col->line.len + 1);
}
memcpy(col->line.data, start.data, col->line.len);
col->line.data[col->line.len] = '\0';
sort: Implement -c and -C flags
2014-11-23 19:35:56 +00:00
}
Add -d, -f and -i flags to sort(1) Here's the patch with updated manpage and usage().
2016-02-16 00:42:25 +00:00
static int
Rename struct linebufline to struct line and add linecmp() This simplifies the handling in sort(1) and comm(1) quite a bit.
2016-03-07 01:04:29 +00:00
skipmodcmp(struct line *a, struct line *b, int flags)
Add -d, -f and -i flags to sort(1) Here's the patch with updated manpage and usage().
2016-02-16 00:42:25 +00:00
{
Rune r1, r2;
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
size_t offa = 0, offb = 0;
Add -d, -f and -i flags to sort(1) Here's the patch with updated manpage and usage().
2016-02-16 00:42:25 +00:00
do {
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
offa += chartorune(&r1, a->data + offa);
offb += chartorune(&r2, b->data + offb);
Add -d, -f and -i flags to sort(1) Here's the patch with updated manpage and usage().
2016-02-16 00:42:25 +00:00
if (flags & MOD_D && flags & MOD_I) {
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
while (offa < a->len && ((!isblankrune(r1) &&
!isalnumrune(r1)) || (!isprintrune(r1))))
offa += chartorune(&r1, a->data + offa);
while (offb < b->len && ((!isblankrune(r2) &&
!isalnumrune(r2)) || (!isprintrune(r2))))
offb += chartorune(&r2, b->data + offb);
Add -d, -f and -i flags to sort(1) Here's the patch with updated manpage and usage().
2016-02-16 00:42:25 +00:00
}
else if (flags & MOD_D) {
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
while (offa < a->len && !isblankrune(r1) &&
!isalnumrune(r1))
offa += chartorune(&r1, a->data + offa);
while (offb < b->len && !isblankrune(r2) &&
!isalnumrune(r2))
offb += chartorune(&r2, b->data + offb);
Add -d, -f and -i flags to sort(1) Here's the patch with updated manpage and usage().
2016-02-16 00:42:25 +00:00
}
else if (flags & MOD_I) {
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
while (offa < a->len && !isprintrune(r1))
offa += chartorune(&r1, a->data + offa);
while (offb < b->len && !isprintrune(r2))
offb += chartorune(&r2, b->data + offb);
Add -d, -f and -i flags to sort(1) Here's the patch with updated manpage and usage().
2016-02-16 00:42:25 +00:00
}
if (flags & MOD_F) {
r1 = toupperrune(r1);
r2 = toupperrune(r2);
}
} while (r1 && r1 == r2);
return r1 - r2;
}
sort: add -k, fix -u Options that are specific to a single key definition are not supported (e.g. "sort -k 2,3n -k 4,4"). Should you try to specify such definitions, sort will return with EXIT_FAILURE and an error message. Instead, all key definitions exclusively use the global settings. It always behaves like -b was set.
2014-04-12 15:53:10 +00:00
static int
Rename struct linebufline to struct line and add linecmp() This simplifies the handling in sort(1) and comm(1) quite a bit.
2016-03-07 01:04:29 +00:00
slinecmp(struct line *a, struct line *b)
add sort
2011-06-02 12:03:34 +00:00
{
sort: add -k, fix -u Options that are specific to a single key definition are not supported (e.g. "sort -k 2,3n -k 4,4"). Should you try to specify such definitions, sort will return with EXIT_FAILURE and an error message. Instead, all key definitions exclusively use the global settings. It always behaves like -b was set.
2014-04-12 15:53:10 +00:00
int res = 0;
sort: Use regular `double` for -n `long double` may require software emulation and the (possible) extra precision is unnecessary here.
2019-02-22 22:26:39 +00:00
double x, y;
Use queue.h-macros in sort(1) This is much easier to read than having yet another handrolled list implementation. Tested and more or less clearly equivalent. Now that I have uni-vac, I'll have enough time to refactor more.
2015-08-02 21:55:54 +00:00
struct keydef *kd;
sort: add -k, fix -u Options that are specific to a single key definition are not supported (e.g. "sort -k 2,3n -k 4,4"). Should you try to specify such definitions, sort will return with EXIT_FAILURE and an error message. Instead, all key definitions exclusively use the global settings. It always behaves like -b was set.
2014-04-12 15:53:10 +00:00
Use queue.h-macros in sort(1) This is much easier to read than having yet another handrolled list implementation. Tested and more or less clearly equivalent. Now that I have uni-vac, I'll have enough time to refactor more.
2015-08-02 21:55:54 +00:00
TAILQ_FOREACH(kd, &kdhead, entry) {
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
columns(a, kd, &col1);
columns(b, kd, &col2);
sort: add -k, fix -u Options that are specific to a single key definition are not supported (e.g. "sort -k 2,3n -k 4,4"). Should you try to specify such definitions, sort will return with EXIT_FAILURE and an error message. Instead, all key definitions exclusively use the global settings. It always behaves like -b was set.
2014-04-12 15:53:10 +00:00
sort: simplify linecmp, rename curr => tail
2014-05-06 16:47:02 +00:00
/* if -u is given, don't use default key definition
* unless it is the only one */
Use queue.h-macros in sort(1) This is much easier to read than having yet another handrolled list implementation. Tested and more or less clearly equivalent. Now that I have uni-vac, I'll have enough time to refactor more.
2015-08-02 21:55:54 +00:00
if (uflag && kd == TAILQ_LAST(&kdhead, kdhead) &&
TAILQ_LAST(&kdhead, kdhead) != TAILQ_FIRST(&kdhead)) {
sort: add -k, fix -u Options that are specific to a single key definition are not supported (e.g. "sort -k 2,3n -k 4,4"). Should you try to specify such definitions, sort will return with EXIT_FAILURE and an error message. Instead, all key definitions exclusively use the global settings. It always behaves like -b was set.
2014-04-12 15:53:10 +00:00
res = 0;
Use queue.h-macros in sort(1) This is much easier to read than having yet another handrolled list implementation. Tested and more or less clearly equivalent. Now that I have uni-vac, I'll have enough time to refactor more.
2015-08-02 21:55:54 +00:00
} else if (kd->flags & MOD_N) {
sort: Use regular `double` for -n `long double` may require software emulation and the (possible) extra precision is unnecessary here.
2019-02-22 22:26:39 +00:00
x = strtod(col1.line.data, NULL);
y = strtod(col2.line.data, NULL);
Audit sort(1) and mark it as finished 1) Remove the function prototypes. No need for them, as the functions are ordered. 2) Add fieldseplen, so the length of the field-separator is not calculated nearly each time skipcolumn() is called. 3) rename next_col to skip_to_next_col so the purpose is clear, also reorder the conditional accordingly. 4) Put parentheses around certain ternary expressions. 5) BUGFIX: Don't just exit() in check(), but make it return something, so we can cleanly fshut() everything. 6) OFF-POSIX: Posix for no apparent reason does not allow more than one file when the -c or -C flags are given. This can be problematic when you want to check multiple files. With the change 5), rewriting check() to return a value, I went off-posix after discussing this with Dimitris to just allow arbitrary numbers of files. Obviously, this does not break scripts and is convenient for everybody who wants to quickly check a big amount of files. As soon as 1 file is "unsorted", the return value is 1, as expected. For convenience reasons, check()'s warning now includes the filename. 7) BUGFIX: Set ret to 2 instead of 1 when the fshut(fp, *argv) fails. 8) BUGFIX: Don't forget to fshut stderr at the end. This would improperly return 1 in the following case: $ sort -c unsorted_file 2> /dev/full 9) Other style changes, line length, empty line before return.
2015-08-04 10:45:59 +00:00
res = (x < y) ? -1 : (x > y);
Add -d, -f and -i flags to sort(1) Here's the patch with updated manpage and usage().
2016-02-16 00:42:25 +00:00
} else if (kd->flags & (MOD_D | MOD_F | MOD_I)) {
sort: Fix line comparison when col buffer contains data from longer line I'm not sure if there are other implications of this or not, but the issue is that columns() uses len to store the allocated buffer size, but linecmp() compares up to len bytes. If those trailing bytes do not match, the line is considered not matching, even though the relevant parts of the buffer do match. To resolve this, also keep track of column capacity. Additionally, since there is no reason to keep the existing data when resizing, just use free and emalloc rather than erealloc. The simplest case I could reduce it to is this: if [ "$(printf '%s\n' a a xxb xxc | ./sort -u)" = "$(printf '%s\n' a xxb xxc)" ] ; then echo pass else echo fail fi
2016-05-15 01:56:54 +00:00
res = skipmodcmp(&col1.line, &col2.line, kd->flags);
sort: support sorting decimal numbers correctly sorry not to have used strtold from the beginning
2015-01-31 18:12:18 +00:00
} else {
sort: Fix line comparison when col buffer contains data from longer line I'm not sure if there are other implications of this or not, but the issue is that columns() uses len to store the allocated buffer size, but linecmp() compares up to len bytes. If those trailing bytes do not match, the line is considered not matching, even though the relevant parts of the buffer do match. To resolve this, also keep track of column capacity. Additionally, since there is no reason to keep the existing data when resizing, just use free and emalloc rather than erealloc. The simplest case I could reduce it to is this: if [ "$(printf '%s\n' a a xxb xxc | ./sort -u)" = "$(printf '%s\n' a xxb xxc)" ] ; then echo pass else echo fail fi
2016-05-15 01:56:54 +00:00
res = linecmp(&col1.line, &col2.line);
sort: support sorting decimal numbers correctly sorry not to have used strtold from the beginning
2015-01-31 18:12:18 +00:00
}
sort: add -k, fix -u Options that are specific to a single key definition are not supported (e.g. "sort -k 2,3n -k 4,4"). Should you try to specify such definitions, sort will return with EXIT_FAILURE and an error message. Instead, all key definitions exclusively use the global settings. It always behaves like -b was set.
2014-04-12 15:53:10 +00:00
Use queue.h-macros in sort(1) This is much easier to read than having yet another handrolled list implementation. Tested and more or less clearly equivalent. Now that I have uni-vac, I'll have enough time to refactor more.
2015-08-02 21:55:54 +00:00
if (kd->flags & MOD_R)
sort: add support for "per-keydef" flags
2014-05-06 14:07:05 +00:00
res = -res;
Use queue.h-macros in sort(1) This is much easier to read than having yet another handrolled list implementation. Tested and more or less clearly equivalent. Now that I have uni-vac, I'll have enough time to refactor more.
2015-08-02 21:55:54 +00:00
if (res)
break;
sort: add -k, fix -u Options that are specific to a single key definition are not supported (e.g. "sort -k 2,3n -k 4,4"). Should you try to specify such definitions, sort will return with EXIT_FAILURE and an error message. Instead, all key definitions exclusively use the global settings. It always behaves like -b was set.
2014-04-12 15:53:10 +00:00
}
Add -m and -o flags to sort(1) Sort comes pretty much automatically, as no script relies on the undefined behaviour of the input _not_ being sorted, we might as well sort the sorted input already. The only downside is memory usage, which can be an issue for large files. The o-flag was trivial to implement.
2015-03-22 22:37:37 +00:00
sort: add support for "per-keydef" flags
2014-05-06 14:07:05 +00:00
return res;
sort: add -k, fix -u Options that are specific to a single key definition are not supported (e.g. "sort -k 2,3n -k 4,4"). Should you try to specify such definitions, sort will return with EXIT_FAILURE and an error message. Instead, all key definitions exclusively use the global settings. It always behaves like -b was set.
2014-04-12 15:53:10 +00:00
}
Audit sort(1) and mark it as finished 1) Remove the function prototypes. No need for them, as the functions are ordered. 2) Add fieldseplen, so the length of the field-separator is not calculated nearly each time skipcolumn() is called. 3) rename next_col to skip_to_next_col so the purpose is clear, also reorder the conditional accordingly. 4) Put parentheses around certain ternary expressions. 5) BUGFIX: Don't just exit() in check(), but make it return something, so we can cleanly fshut() everything. 6) OFF-POSIX: Posix for no apparent reason does not allow more than one file when the -c or -C flags are given. This can be problematic when you want to check multiple files. With the change 5), rewriting check() to return a value, I went off-posix after discussing this with Dimitris to just allow arbitrary numbers of files. Obviously, this does not break scripts and is convenient for everybody who wants to quickly check a big amount of files. As soon as 1 file is "unsorted", the return value is 1, as expected. For convenience reasons, check()'s warning now includes the filename. 7) BUGFIX: Set ret to 2 instead of 1 when the fshut(fp, *argv) fails. 8) BUGFIX: Don't forget to fshut stderr at the end. This would improperly return 1 in the following case: $ sort -c unsorted_file 2> /dev/full 9) Other style changes, line length, empty line before return.
2015-08-04 10:45:59 +00:00
static int
check(FILE *fp, const char *fname)
sort: add -k, fix -u Options that are specific to a single key definition are not supported (e.g. "sort -k 2,3n -k 4,4"). Should you try to specify such definitions, sort will return with EXIT_FAILURE and an error message. Instead, all key definitions exclusively use the global settings. It always behaves like -b was set.
2014-04-12 15:53:10 +00:00
{
Rename struct linebufline to struct line and add linecmp() This simplifies the handling in sort(1) and comm(1) quite a bit.
2016-03-07 01:04:29 +00:00
static struct line prev, cur, tmp;
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
static size_t prevsize, cursize, tmpsize;
sort: Fix -c option In eb9bda878736344d1bef06d42e57e96de542a663, a bug was introduced in the handling of -1 return values from getline. Since the type of the len field in struct line is unsigned, the break condition was never true. This caused sort -c to never succeed.
2016-03-12 19:46:31 +00:00
ssize_t len;
sort: add -k, fix -u Options that are specific to a single key definition are not supported (e.g. "sort -k 2,3n -k 4,4"). Should you try to specify such definitions, sort will return with EXIT_FAILURE and an error message. Instead, all key definitions exclusively use the global settings. It always behaves like -b was set.
2014-04-12 15:53:10 +00:00
sort: Fix -c option In eb9bda878736344d1bef06d42e57e96de542a663, a bug was introduced in the handling of -1 return values from getline. Since the type of the len field in struct line is unsigned, the break condition was never true. This caused sort -c to never succeed.
2016-03-12 19:46:31 +00:00
if (!prev.data) {
if ((len = getline(&prev.data, &prevsize, fp)) < 0)
eprintf("getline:");
prev.len = len;
}
while ((len = getline(&cur.data, &cursize, fp)) > 0) {
cur.len = len;
Rename linecmp and line-structs in join(1) and sort(1) We will steal the names for the global functions.
2016-03-07 00:58:31 +00:00
if (uflag > slinecmp(&cur, &prev)) {
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
if (!Cflag) {
weprintf("disorder %s: ", fname);
fwrite(cur.data, 1, cur.len, stderr);
}
Audit sort(1) and mark it as finished 1) Remove the function prototypes. No need for them, as the functions are ordered. 2) Add fieldseplen, so the length of the field-separator is not calculated nearly each time skipcolumn() is called. 3) rename next_col to skip_to_next_col so the purpose is clear, also reorder the conditional accordingly. 4) Put parentheses around certain ternary expressions. 5) BUGFIX: Don't just exit() in check(), but make it return something, so we can cleanly fshut() everything. 6) OFF-POSIX: Posix for no apparent reason does not allow more than one file when the -c or -C flags are given. This can be problematic when you want to check multiple files. With the change 5), rewriting check() to return a value, I went off-posix after discussing this with Dimitris to just allow arbitrary numbers of files. Obviously, this does not break scripts and is convenient for everybody who wants to quickly check a big amount of files. As soon as 1 file is "unsorted", the return value is 1, as expected. For convenience reasons, check()'s warning now includes the filename. 7) BUGFIX: Set ret to 2 instead of 1 when the fshut(fp, *argv) fails. 8) BUGFIX: Don't forget to fshut stderr at the end. This would improperly return 1 in the following case: $ sort -c unsorted_file 2> /dev/full 9) Other style changes, line length, empty line before return.
2015-08-04 10:45:59 +00:00
return 1;
sort: work with signed integers as well
2014-05-06 11:35:06 +00:00
}
Factor out parse_keydef() into addkeydef() and reorder functions Add a small comment explaining the data-structure and sort the functions according to usage, not alphabetically.
2015-08-02 23:12:29 +00:00
tmp = cur;
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
tmpsize = cursize;
Factor out parse_keydef() into addkeydef() and reorder functions Add a small comment explaining the data-structure and sort the functions according to usage, not alphabetically.
2015-08-02 23:12:29 +00:00
cur = prev;
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
cursize = prevsize;
Factor out parse_keydef() into addkeydef() and reorder functions Add a small comment explaining the data-structure and sort the functions according to usage, not alphabetically.
2015-08-02 23:12:29 +00:00
prev = tmp;
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
prevsize = tmpsize;
Add -n support to sort(1)
2013-12-12 13:08:49 +00:00
}
Audit sort(1) and mark it as finished 1) Remove the function prototypes. No need for them, as the functions are ordered. 2) Add fieldseplen, so the length of the field-separator is not calculated nearly each time skipcolumn() is called. 3) rename next_col to skip_to_next_col so the purpose is clear, also reorder the conditional accordingly. 4) Put parentheses around certain ternary expressions. 5) BUGFIX: Don't just exit() in check(), but make it return something, so we can cleanly fshut() everything. 6) OFF-POSIX: Posix for no apparent reason does not allow more than one file when the -c or -C flags are given. This can be problematic when you want to check multiple files. With the change 5), rewriting check() to return a value, I went off-posix after discussing this with Dimitris to just allow arbitrary numbers of files. Obviously, this does not break scripts and is convenient for everybody who wants to quickly check a big amount of files. As soon as 1 file is "unsorted", the return value is 1, as expected. For convenience reasons, check()'s warning now includes the filename. 7) BUGFIX: Set ret to 2 instead of 1 when the fshut(fp, *argv) fails. 8) BUGFIX: Don't forget to fshut stderr at the end. This would improperly return 1 in the following case: $ sort -c unsorted_file 2> /dev/full 9) Other style changes, line length, empty line before return.
2015-08-04 10:45:59 +00:00
return 0;
add sort
2011-06-02 12:03:34 +00:00
}
Eliminating the getopt disgrace.
2013-06-14 18:20:47 +00:00
Reorder functions in sort(1) I kind of missed that the sorting was still not properly done. parse_flags() and addkeydef() are independent of everything else, so they can be put at the bottom. Sorting the other functions reveals the true hierarchy much better.
2015-08-02 23:32:21 +00:00
static int
parse_flags(char **s, int *flags, int bflag)
sort: add -k, fix -u Options that are specific to a single key definition are not supported (e.g. "sort -k 2,3n -k 4,4"). Should you try to specify such definitions, sort will return with EXIT_FAILURE and an error message. Instead, all key definitions exclusively use the global settings. It always behaves like -b was set.
2014-04-12 15:53:10 +00:00
{
Reorder functions in sort(1) I kind of missed that the sorting was still not properly done. parse_flags() and addkeydef() are independent of everything else, so they can be put at the bottom. Sorting the other functions reveals the true hierarchy much better.
2015-08-02 23:32:21 +00:00
while (isalpha((int)**s)) {
switch (*((*s)++)) {
case 'b':
*flags |= bflag;
break;
Add -d, -f and -i flags to sort(1) Here's the patch with updated manpage and usage().
2016-02-16 00:42:25 +00:00
case 'd':
*flags |= MOD_D;
break;
case 'f':
*flags |= MOD_F;
break;
case 'i':
*flags |= MOD_I;
break;
Reorder functions in sort(1) I kind of missed that the sorting was still not properly done. parse_flags() and addkeydef() are independent of everything else, so they can be put at the bottom. Sorting the other functions reveals the true hierarchy much better.
2015-08-02 23:32:21 +00:00
case 'n':
*flags |= MOD_N;
break;
case 'r':
*flags |= MOD_R;
break;
default:
return -1;
}
}
Add -m and -o flags to sort(1) Sort comes pretty much automatically, as no script relies on the undefined behaviour of the input _not_ being sorted, we might as well sort the sorted input already. The only downside is memory usage, which can be an issue for large files. The o-flag was trivial to implement.
2015-03-22 22:37:37 +00:00
Reorder functions in sort(1) I kind of missed that the sorting was still not properly done. parse_flags() and addkeydef() are independent of everything else, so they can be put at the bottom. Sorting the other functions reveals the true hierarchy much better.
2015-08-02 23:32:21 +00:00
return 0;
sort: add -k, fix -u Options that are specific to a single key definition are not supported (e.g. "sort -k 2,3n -k 4,4"). Should you try to specify such definitions, sort will return with EXIT_FAILURE and an error message. Instead, all key definitions exclusively use the global settings. It always behaves like -b was set.
2014-04-12 15:53:10 +00:00
}
Reorder functions in sort(1) I kind of missed that the sorting was still not properly done. parse_flags() and addkeydef() are independent of everything else, so they can be put at the bottom. Sorting the other functions reveals the true hierarchy much better.
2015-08-02 23:32:21 +00:00
static void
addkeydef(char *kdstr, int flags)
sort: add -k, fix -u Options that are specific to a single key definition are not supported (e.g. "sort -k 2,3n -k 4,4"). Should you try to specify such definitions, sort will return with EXIT_FAILURE and an error message. Instead, all key definitions exclusively use the global settings. It always behaves like -b was set.
2014-04-12 15:53:10 +00:00
{
Reorder functions in sort(1) I kind of missed that the sorting was still not properly done. parse_flags() and addkeydef() are independent of everything else, so they can be put at the bottom. Sorting the other functions reveals the true hierarchy much better.
2015-08-02 23:32:21 +00:00
struct keydef *kd;
sort: don't repeat skipping columns logic
2014-05-03 16:34:51 +00:00
Reorder functions in sort(1) I kind of missed that the sorting was still not properly done. parse_flags() and addkeydef() are independent of everything else, so they can be put at the bottom. Sorting the other functions reveals the true hierarchy much better.
2015-08-02 23:32:21 +00:00
kd = enmalloc(2, sizeof(*kd));
Minor style changes to sort
2014-04-18 16:21:31 +00:00
Reorder functions in sort(1) I kind of missed that the sorting was still not properly done. parse_flags() and addkeydef() are independent of everything else, so they can be put at the bottom. Sorting the other functions reveals the true hierarchy much better.
2015-08-02 23:32:21 +00:00
/* parse key definition kdstr with format
* start_column[.start_char][flags][,end_column[.end_char][flags]]
*/
kd->start_column = 1;
kd->start_char = 1;
kd->end_column = 0; /* 0 means end of line */
kd->end_char = 0; /* 0 means end of column */
kd->flags = flags;
sort: add -k, fix -u Options that are specific to a single key definition are not supported (e.g. "sort -k 2,3n -k 4,4"). Should you try to specify such definitions, sort will return with EXIT_FAILURE and an error message. Instead, all key definitions exclusively use the global settings. It always behaves like -b was set.
2014-04-12 15:53:10 +00:00
Reorder functions in sort(1) I kind of missed that the sorting was still not properly done. parse_flags() and addkeydef() are independent of everything else, so they can be put at the bottom. Sorting the other functions reveals the true hierarchy much better.
2015-08-02 23:32:21 +00:00
if ((kd->start_column = strtol(kdstr, &kdstr, 10)) < 1)
enprintf(2, "invalid start column in key definition\n");
if (*kdstr == '.') {
if ((kd->start_char = strtol(kdstr + 1, &kdstr, 10)) < 1)
Audit sort(1) and mark it as finished 1) Remove the function prototypes. No need for them, as the functions are ordered. 2) Add fieldseplen, so the length of the field-separator is not calculated nearly each time skipcolumn() is called. 3) rename next_col to skip_to_next_col so the purpose is clear, also reorder the conditional accordingly. 4) Put parentheses around certain ternary expressions. 5) BUGFIX: Don't just exit() in check(), but make it return something, so we can cleanly fshut() everything. 6) OFF-POSIX: Posix for no apparent reason does not allow more than one file when the -c or -C flags are given. This can be problematic when you want to check multiple files. With the change 5), rewriting check() to return a value, I went off-posix after discussing this with Dimitris to just allow arbitrary numbers of files. Obviously, this does not break scripts and is convenient for everybody who wants to quickly check a big amount of files. As soon as 1 file is "unsorted", the return value is 1, as expected. For convenience reasons, check()'s warning now includes the filename. 7) BUGFIX: Set ret to 2 instead of 1 when the fshut(fp, *argv) fails. 8) BUGFIX: Don't forget to fshut stderr at the end. This would improperly return 1 in the following case: $ sort -c unsorted_file 2> /dev/full 9) Other style changes, line length, empty line before return.
2015-08-04 10:45:59 +00:00
enprintf(2, "invalid start character in key "
"definition\n");
sort: don't repeat skipping columns logic
2014-05-03 16:34:51 +00:00
}
Reorder functions in sort(1) I kind of missed that the sorting was still not properly done. parse_flags() and addkeydef() are independent of everything else, so they can be put at the bottom. Sorting the other functions reveals the true hierarchy much better.
2015-08-02 23:32:21 +00:00
if (parse_flags(&kdstr, &kd->flags, MOD_STARTB) < 0)
enprintf(2, "invalid start flags in key definition\n");
if (*kdstr == ',') {
if ((kd->end_column = strtol(kdstr + 1, &kdstr, 10)) < 0)
enprintf(2, "invalid end column in key definition\n");
if (*kdstr == '.') {
if ((kd->end_char = strtol(kdstr + 1, &kdstr, 10)) < 0)
Audit sort(1) and mark it as finished 1) Remove the function prototypes. No need for them, as the functions are ordered. 2) Add fieldseplen, so the length of the field-separator is not calculated nearly each time skipcolumn() is called. 3) rename next_col to skip_to_next_col so the purpose is clear, also reorder the conditional accordingly. 4) Put parentheses around certain ternary expressions. 5) BUGFIX: Don't just exit() in check(), but make it return something, so we can cleanly fshut() everything. 6) OFF-POSIX: Posix for no apparent reason does not allow more than one file when the -c or -C flags are given. This can be problematic when you want to check multiple files. With the change 5), rewriting check() to return a value, I went off-posix after discussing this with Dimitris to just allow arbitrary numbers of files. Obviously, this does not break scripts and is convenient for everybody who wants to quickly check a big amount of files. As soon as 1 file is "unsorted", the return value is 1, as expected. For convenience reasons, check()'s warning now includes the filename. 7) BUGFIX: Set ret to 2 instead of 1 when the fshut(fp, *argv) fails. 8) BUGFIX: Don't forget to fshut stderr at the end. This would improperly return 1 in the following case: $ sort -c unsorted_file 2> /dev/full 9) Other style changes, line length, empty line before return.
2015-08-04 10:45:59 +00:00
enprintf(2, "invalid end character in key "
"definition\n");
Reorder functions in sort(1) I kind of missed that the sorting was still not properly done. parse_flags() and addkeydef() are independent of everything else, so they can be put at the bottom. Sorting the other functions reveals the true hierarchy much better.
2015-08-02 23:32:21 +00:00
}
if (parse_flags(&kdstr, &kd->flags, MOD_ENDB) < 0)
enprintf(2, "invalid end flags in key definition\n");
}
if (*kdstr != '\0')
enprintf(2, "invalid key definition\n");
TAILQ_INSERT_TAIL(&kdhead, kd, entry);
sort: add -k, fix -u Options that are specific to a single key definition are not supported (e.g. "sort -k 2,3n -k 4,4"). Should you try to specify such definitions, sort will return with EXIT_FAILURE and an error message. Instead, all key definitions exclusively use the global settings. It always behaves like -b was set.
2014-04-12 15:53:10 +00:00
}
grep, kill, renice, sort: style: put main at bottom
2015-03-07 14:39:39 +00:00
static void
usage(void)
{
Add -d, -f and -i flags to sort(1) Here's the patch with updated manpage and usage().
2016-02-16 00:42:25 +00:00
enprintf(2, "usage: %s [-Cbcdfimnru] [-o outfile] [-t delim] "
Factor out parse_keydef() into addkeydef() and reorder functions Add a small comment explaining the data-structure and sort the functions according to usage, not alphabetically.
2015-08-02 23:12:29 +00:00
"[-k def]... [file ...]\n", argv0);
grep, kill, renice, sort: style: put main at bottom
2015-03-07 14:39:39 +00:00
}
int
main(int argc, char *argv[])
{
Add -m and -o flags to sort(1) Sort comes pretty much automatically, as no script relies on the undefined behaviour of the input _not_ being sorted, we might as well sort the sorted input already. The only downside is memory usage, which can be an issue for large files. The o-flag was trivial to implement.
2015-03-22 22:37:37 +00:00
FILE *fp, *ofp = stdout;
grep, kill, renice, sort: style: put main at bottom
2015-03-07 14:39:39 +00:00
struct linebuf linebuf = EMPTY_LINEBUF;
Add -m and -o flags to sort(1) Sort comes pretty much automatically, as no script relies on the undefined behaviour of the input _not_ being sorted, we might as well sort the sorted input already. The only downside is memory usage, which can be an issue for large files. The o-flag was trivial to implement.
2015-03-22 22:37:37 +00:00
size_t i;
Add *fshut() functions to properly flush file streams This has been a known issue for a long time. Example: printf "word" > /dev/full wouldn't report there's not enough space on the device. This is due to the fact that every libc has internal buffers for stdout which store fragments of written data until they reach a certain size or on some callback to flush them all at once to the kernel. You can force the libc to flush them with fflush(). In case flushing fails, you can check the return value of fflush() and report an error. However, previously, sbase didn't have such checks and without fflush(), the libc silently flushes the buffers on exit without checking the errors. No offense, but there's no way for the libc to report errors in the exit- condition. GNU coreutils solve this by having onexit-callbacks to handle the flushing and report issues, but they have obvious deficiencies. After long discussions on IRC, we came to the conclusion that checking the return value of every io-function would be a bit too much, and having a general-purpose fclose-wrapper would be the best way to go. It turned out that fclose() alone is not enough to detect errors. The right way to do it is to fflush() + check ferror on the fp and then to a fclose(). This is what fshut does and that's how it's done before each return. The return value is obviously affected, reporting an error in case a flush or close failed, but also when reading failed for some reason, the error- state is caught. the !!( ... + ...) construction is used to call all functions inside the brackets and not "terminating" on the first. We want errors to be reported, but there's no reason to stop flushing buffers when one other file buffer has issues. Obviously, functionales come before the flush and ret-logic comes after to prevent early exits as well without reporting warnings if there are any. One more advantage of fshut() is that it is even able to report errors on obscure NFS-setups which the other coreutils are unable to detect, because they only check the return-value of fflush() and fclose(), not ferror() as well.
2015-04-04 19:25:17 +00:00
int global_flags = 0, ret = 0;
Add -m and -o flags to sort(1) Sort comes pretty much automatically, as no script relies on the undefined behaviour of the input _not_ being sorted, we might as well sort the sorted input already. The only downside is memory usage, which can be an issue for large files. The o-flag was trivial to implement.
2015-03-22 22:37:37 +00:00
char *outfile = NULL;
grep, kill, renice, sort: style: put main at bottom
2015-03-07 14:39:39 +00:00
ARGBEGIN {
case 'C':
Cflag = 1;
break;
case 'b':
global_flags |= MOD_STARTB | MOD_ENDB;
break;
case 'c':
cflag = 1;
break;
Add -d, -f and -i flags to sort(1) Here's the patch with updated manpage and usage().
2016-02-16 00:42:25 +00:00
case 'd':
global_flags |= MOD_D;
break;
case 'f':
global_flags |= MOD_F;
break;
case 'i':
global_flags |= MOD_I;
break;
grep, kill, renice, sort: style: put main at bottom
2015-03-07 14:39:39 +00:00
case 'k':
addkeydef(EARGF(usage()), global_flags);
break;
Add -m and -o flags to sort(1) Sort comes pretty much automatically, as no script relies on the undefined behaviour of the input _not_ being sorted, we might as well sort the sorted input already. The only downside is memory usage, which can be an issue for large files. The o-flag was trivial to implement.
2015-03-22 22:37:37 +00:00
case 'm':
sort: reuse buffer in columns() speeds up sorting for huge input aswell.
2015-05-05 11:41:43 +00:00
/* more or less for free, but for performance-reasons,
Add -m and -o flags to sort(1) Sort comes pretty much automatically, as no script relies on the undefined behaviour of the input _not_ being sorted, we might as well sort the sorted input already. The only downside is memory usage, which can be an issue for large files. The o-flag was trivial to implement.
2015-03-22 22:37:37 +00:00
* we should keep this flag in mind and maybe some later
* day implement it properly so we don't run out of memory
* while merging large sorted files.
*/
break;
grep, kill, renice, sort: style: put main at bottom
2015-03-07 14:39:39 +00:00
case 'n':
global_flags |= MOD_N;
break;
Add -m and -o flags to sort(1) Sort comes pretty much automatically, as no script relies on the undefined behaviour of the input _not_ being sorted, we might as well sort the sorted input already. The only downside is memory usage, which can be an issue for large files. The o-flag was trivial to implement.
2015-03-22 22:37:37 +00:00
case 'o':
outfile = EARGF(usage());
break;
grep, kill, renice, sort: style: put main at bottom
2015-03-07 14:39:39 +00:00
case 'r':
global_flags |= MOD_R;
break;
case 't':
fieldsep = EARGF(usage());
Error out when giving an empty delimiter to sort(1)
2016-03-07 10:18:30 +00:00
if (!*fieldsep)
eprintf("empty delimiter\n");
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
fieldseplen = unescape(fieldsep);
grep, kill, renice, sort: style: put main at bottom
2015-03-07 14:39:39 +00:00
break;
case 'u':
uflag = 1;
break;
default:
usage();
No need for semicolon after ARGEND This is also the style used in Plan 9.
2015-11-01 10:16:49 +00:00
} ARGEND
grep, kill, renice, sort: style: put main at bottom
2015-03-07 14:39:39 +00:00
sort: apply -b only to "custom" keys
2015-04-05 18:31:28 +00:00
/* -b shall only apply to custom key definitions */
Use queue.h-macros in sort(1) This is much easier to read than having yet another handrolled list implementation. Tested and more or less clearly equivalent. Now that I have uni-vac, I'll have enough time to refactor more.
2015-08-02 21:55:54 +00:00
if (TAILQ_EMPTY(&kdhead) && global_flags)
addkeydef("1", global_flags & ~(MOD_STARTB | MOD_ENDB));
sort: Don't do fallback top-level sort in check mode The fallback useful to provide a consistent order of tied lines, but in check mode, we don't want it to report disorder for equal lines (according to the passed flags). Thanks to Richard Ipsum for the bug report and proposed patch.
2020-01-03 23:32:07 +00:00
if (TAILQ_EMPTY(&kdhead) || (!Cflag && !cflag))
addkeydef("1", global_flags & MOD_R);
grep, kill, renice, sort: style: put main at bottom
2015-03-07 14:39:39 +00:00
Add -m and -o flags to sort(1) Sort comes pretty much automatically, as no script relies on the undefined behaviour of the input _not_ being sorted, we might as well sort the sorted input already. The only downside is memory usage, which can be an issue for large files. The o-flag was trivial to implement.
2015-03-22 22:37:37 +00:00
if (!argc) {
grep, kill, renice, sort: style: put main at bottom
2015-03-07 14:39:39 +00:00
if (Cflag || cflag) {
Audit sort(1) and mark it as finished 1) Remove the function prototypes. No need for them, as the functions are ordered. 2) Add fieldseplen, so the length of the field-separator is not calculated nearly each time skipcolumn() is called. 3) rename next_col to skip_to_next_col so the purpose is clear, also reorder the conditional accordingly. 4) Put parentheses around certain ternary expressions. 5) BUGFIX: Don't just exit() in check(), but make it return something, so we can cleanly fshut() everything. 6) OFF-POSIX: Posix for no apparent reason does not allow more than one file when the -c or -C flags are given. This can be problematic when you want to check multiple files. With the change 5), rewriting check() to return a value, I went off-posix after discussing this with Dimitris to just allow arbitrary numbers of files. Obviously, this does not break scripts and is convenient for everybody who wants to quickly check a big amount of files. As soon as 1 file is "unsorted", the return value is 1, as expected. For convenience reasons, check()'s warning now includes the filename. 7) BUGFIX: Set ret to 2 instead of 1 when the fshut(fp, *argv) fails. 8) BUGFIX: Don't forget to fshut stderr at the end. This would improperly return 1 in the following case: $ sort -c unsorted_file 2> /dev/full 9) Other style changes, line length, empty line before return.
2015-08-04 10:45:59 +00:00
if (check(stdin, "<stdin>") && !ret)
ret = 1;
grep, kill, renice, sort: style: put main at bottom
2015-03-07 14:39:39 +00:00
} else {
getlines(stdin, &linebuf);
}
Add -m and -o flags to sort(1) Sort comes pretty much automatically, as no script relies on the undefined behaviour of the input _not_ being sorted, we might as well sort the sorted input already. The only downside is memory usage, which can be an issue for large files. The o-flag was trivial to implement.
2015-03-22 22:37:37 +00:00
} else for (; *argv; argc--, argv++) {
Remove handrolled strcmp()'s Favor readability over bare-metal.
2015-05-19 15:44:15 +00:00
if (!strcmp(*argv, "-")) {
Handle '-' consistently In general, POSIX does not define /dev/std{in, out, err} because it does not want to depend on the dev-filesystem. For utilities, it thus introduced the '-'-keyword to denote standard input (and output in some cases) and the programs have to deal with it accordingly. Sadly, the design of many tools doesn't allow strict shell-redirections and many scripts don't even use this feature when possible. Thus, we made the decision to implement it consistently across all tools where it makes sense (namely those which read files). Along the way, I spotted some behavioural bugs in libutil/crypt.c and others where it was forgotten to fshut the files after use.
2015-05-15 11:28:39 +00:00
*argv = "<stdin>";
fp = stdin;
} else if (!(fp = fopen(*argv, "r"))) {
Add -m and -o flags to sort(1) Sort comes pretty much automatically, as no script relies on the undefined behaviour of the input _not_ being sorted, we might as well sort the sorted input already. The only downside is memory usage, which can be an issue for large files. The o-flag was trivial to implement.
2015-03-22 22:37:37 +00:00
enprintf(2, "fopen %s:", *argv);
grep, kill, renice, sort: style: put main at bottom
2015-03-07 14:39:39 +00:00
continue;
}
if (Cflag || cflag) {
Audit sort(1) and mark it as finished 1) Remove the function prototypes. No need for them, as the functions are ordered. 2) Add fieldseplen, so the length of the field-separator is not calculated nearly each time skipcolumn() is called. 3) rename next_col to skip_to_next_col so the purpose is clear, also reorder the conditional accordingly. 4) Put parentheses around certain ternary expressions. 5) BUGFIX: Don't just exit() in check(), but make it return something, so we can cleanly fshut() everything. 6) OFF-POSIX: Posix for no apparent reason does not allow more than one file when the -c or -C flags are given. This can be problematic when you want to check multiple files. With the change 5), rewriting check() to return a value, I went off-posix after discussing this with Dimitris to just allow arbitrary numbers of files. Obviously, this does not break scripts and is convenient for everybody who wants to quickly check a big amount of files. As soon as 1 file is "unsorted", the return value is 1, as expected. For convenience reasons, check()'s warning now includes the filename. 7) BUGFIX: Set ret to 2 instead of 1 when the fshut(fp, *argv) fails. 8) BUGFIX: Don't forget to fshut stderr at the end. This would improperly return 1 in the following case: $ sort -c unsorted_file 2> /dev/full 9) Other style changes, line length, empty line before return.
2015-08-04 10:45:59 +00:00
if (check(fp, *argv) && !ret)
ret = 1;
grep, kill, renice, sort: style: put main at bottom
2015-03-07 14:39:39 +00:00
} else {
getlines(fp, &linebuf);
}
Handle '-' consistently In general, POSIX does not define /dev/std{in, out, err} because it does not want to depend on the dev-filesystem. For utilities, it thus introduced the '-'-keyword to denote standard input (and output in some cases) and the programs have to deal with it accordingly. Sadly, the design of many tools doesn't allow strict shell-redirections and many scripts don't even use this feature when possible. Thus, we made the decision to implement it consistently across all tools where it makes sense (namely those which read files). Along the way, I spotted some behavioural bugs in libutil/crypt.c and others where it was forgotten to fshut the files after use.
2015-05-15 11:28:39 +00:00
if (fp != stdin && fshut(fp, *argv))
Audit sort(1) and mark it as finished 1) Remove the function prototypes. No need for them, as the functions are ordered. 2) Add fieldseplen, so the length of the field-separator is not calculated nearly each time skipcolumn() is called. 3) rename next_col to skip_to_next_col so the purpose is clear, also reorder the conditional accordingly. 4) Put parentheses around certain ternary expressions. 5) BUGFIX: Don't just exit() in check(), but make it return something, so we can cleanly fshut() everything. 6) OFF-POSIX: Posix for no apparent reason does not allow more than one file when the -c or -C flags are given. This can be problematic when you want to check multiple files. With the change 5), rewriting check() to return a value, I went off-posix after discussing this with Dimitris to just allow arbitrary numbers of files. Obviously, this does not break scripts and is convenient for everybody who wants to quickly check a big amount of files. As soon as 1 file is "unsorted", the return value is 1, as expected. For convenience reasons, check()'s warning now includes the filename. 7) BUGFIX: Set ret to 2 instead of 1 when the fshut(fp, *argv) fails. 8) BUGFIX: Don't forget to fshut stderr at the end. This would improperly return 1 in the following case: $ sort -c unsorted_file 2> /dev/full 9) Other style changes, line length, empty line before return.
2015-08-04 10:45:59 +00:00
ret = 2;
grep, kill, renice, sort: style: put main at bottom
2015-03-07 14:39:39 +00:00
}
if (!Cflag && !cflag) {
Add -m and -o flags to sort(1) Sort comes pretty much automatically, as no script relies on the undefined behaviour of the input _not_ being sorted, we might as well sort the sorted input already. The only downside is memory usage, which can be an issue for large files. The o-flag was trivial to implement.
2015-03-22 22:37:37 +00:00
if (outfile && !(ofp = fopen(outfile, "w")))
eprintf("fopen %s:", outfile);
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
qsort(linebuf.lines, linebuf.nlines, sizeof(*linebuf.lines),
Rename linecmp and line-structs in join(1) and sort(1) We will steal the names for the global functions.
2016-03-07 00:58:31 +00:00
(int (*)(const void *, const void *))slinecmp);
grep, kill, renice, sort: style: put main at bottom
2015-03-07 14:39:39 +00:00
for (i = 0; i < linebuf.nlines; i++) {
Audit sort(1) and mark it as finished 1) Remove the function prototypes. No need for them, as the functions are ordered. 2) Add fieldseplen, so the length of the field-separator is not calculated nearly each time skipcolumn() is called. 3) rename next_col to skip_to_next_col so the purpose is clear, also reorder the conditional accordingly. 4) Put parentheses around certain ternary expressions. 5) BUGFIX: Don't just exit() in check(), but make it return something, so we can cleanly fshut() everything. 6) OFF-POSIX: Posix for no apparent reason does not allow more than one file when the -c or -C flags are given. This can be problematic when you want to check multiple files. With the change 5), rewriting check() to return a value, I went off-posix after discussing this with Dimitris to just allow arbitrary numbers of files. Obviously, this does not break scripts and is convenient for everybody who wants to quickly check a big amount of files. As soon as 1 file is "unsorted", the return value is 1, as expected. For convenience reasons, check()'s warning now includes the filename. 7) BUGFIX: Set ret to 2 instead of 1 when the fshut(fp, *argv) fails. 8) BUGFIX: Don't forget to fshut stderr at the end. This would improperly return 1 in the following case: $ sort -c unsorted_file 2> /dev/full 9) Other style changes, line length, empty line before return.
2015-08-04 10:45:59 +00:00
if (!uflag || i == 0 ||
Rename linecmp and line-structs in join(1) and sort(1) We will steal the names for the global functions.
2016-03-07 00:58:31 +00:00
slinecmp(&linebuf.lines[i], &linebuf.lines[i - 1])) {
Support NUL-containing lines in sort(1) For sort(1) we need memmem(), which I imported from OpenBSD. Inside sort(1), the changes involved working with the explicit lengths given by getlines() earlier and rewriting some of the functions. Now we can handle NUL-characters in the input just fine.
2016-02-28 23:47:10 +00:00
fwrite(linebuf.lines[i].data, 1,
linebuf.lines[i].len, ofp);
grep, kill, renice, sort: style: put main at bottom
2015-03-07 14:39:39 +00:00
}
}
}
Audit sort(1) and mark it as finished 1) Remove the function prototypes. No need for them, as the functions are ordered. 2) Add fieldseplen, so the length of the field-separator is not calculated nearly each time skipcolumn() is called. 3) rename next_col to skip_to_next_col so the purpose is clear, also reorder the conditional accordingly. 4) Put parentheses around certain ternary expressions. 5) BUGFIX: Don't just exit() in check(), but make it return something, so we can cleanly fshut() everything. 6) OFF-POSIX: Posix for no apparent reason does not allow more than one file when the -c or -C flags are given. This can be problematic when you want to check multiple files. With the change 5), rewriting check() to return a value, I went off-posix after discussing this with Dimitris to just allow arbitrary numbers of files. Obviously, this does not break scripts and is convenient for everybody who wants to quickly check a big amount of files. As soon as 1 file is "unsorted", the return value is 1, as expected. For convenience reasons, check()'s warning now includes the filename. 7) BUGFIX: Set ret to 2 instead of 1 when the fshut(fp, *argv) fails. 8) BUGFIX: Don't forget to fshut stderr at the end. This would improperly return 1 in the following case: $ sort -c unsorted_file 2> /dev/full 9) Other style changes, line length, empty line before return.
2015-08-04 10:45:59 +00:00
if (fshut(stdin, "<stdin>") | fshut(stdout, "<stdout>") |
fshut(stderr, "<stderr>"))
Simplify return & fshut() logic Get rid of the !!()-constructs and use ret where available (or introduce it). In some cases, there would be an "abort" on the first fshut-error, but we want to close all files and report all warnings and then quit, not just the warning for the first file.
2015-05-24 23:33:19 +00:00
ret = 2;
Add *fshut() functions to properly flush file streams This has been a known issue for a long time. Example: printf "word" > /dev/full wouldn't report there's not enough space on the device. This is due to the fact that every libc has internal buffers for stdout which store fragments of written data until they reach a certain size or on some callback to flush them all at once to the kernel. You can force the libc to flush them with fflush(). In case flushing fails, you can check the return value of fflush() and report an error. However, previously, sbase didn't have such checks and without fflush(), the libc silently flushes the buffers on exit without checking the errors. No offense, but there's no way for the libc to report errors in the exit- condition. GNU coreutils solve this by having onexit-callbacks to handle the flushing and report issues, but they have obvious deficiencies. After long discussions on IRC, we came to the conclusion that checking the return value of every io-function would be a bit too much, and having a general-purpose fclose-wrapper would be the best way to go. It turned out that fclose() alone is not enough to detect errors. The right way to do it is to fflush() + check ferror on the fp and then to a fclose(). This is what fshut does and that's how it's done before each return. The return value is obviously affected, reporting an error in case a flush or close failed, but also when reading failed for some reason, the error- state is caught. the !!( ... + ...) construction is used to call all functions inside the brackets and not "terminating" on the first. We want errors to be reported, but there's no reason to stop flushing buffers when one other file buffer has issues. Obviously, functionales come before the flush and ret-logic comes after to prevent early exits as well without reporting warnings if there are any. One more advantage of fshut() is that it is even able to report errors on obscure NFS-setups which the other coreutils are unable to detect, because they only check the return-value of fflush() and fclose(), not ferror() as well.
2015-04-04 19:25:17 +00:00
return ret;
grep, kill, renice, sort: style: put main at bottom
2015-03-07 14:39:39 +00:00
}
Reference in New Issue Copy Permalink